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INTRODUCTION 


Structure  of  this  Report 

The  research  reported  here  was  undertaken  to  develop  a  concept  for  representing  the  products  of 
a  Cognitive  Work  Analysis  in  a  comprehensive,  integrated  fashion  that  would  better  support  the 
application  of  Cognitive  Systems  Engineering  practice  within  the  systems  acquisition  process. 
This  report  first  provides  some  background  regarding  the  problem  area  and  the  scope  and 
objectives  of  the  research.  It  then  introduces  the  subject  of  an  information  environment  to 
support  system  design,  leading  up  to  a  vision  for  an  information  structure — a  “reasoning  space.” 
This  report  then  illustrates  how  a  human-systems  designer  or  analyst  might  use  the  reasoning 
space  to  understand  a  work  domain  using  the  context  of  a  Time  Sensitive  Targeting  cell  in  an  Air 
Operations  Center. 

Appendices  A  and  B  provide  some  tutorial  material  regarding  the  Work  Domain  Analysis  phase 
of  a  Cognitive  Work  Analysis.  Representational  issues  and  forms  are  the  topic  of  Appendix  C. 
The  relationship  of  Work  Domain  Analysis  to  traditional  systems  engineering  analyses  is  the 
topic  of  Appendix  D.  Appendices  E  through  J  document  some  of  the  exchanges  Dr.  Lintem  had 
with  Subject  Matter  Experts  while  conducting  the  research  and  exploring  ideas  and  approaches 
for  this  project,  and  Appendix  K  concludes  with  some  thoughts  stimulated  by  those  exchanges. 

Background 

Cognitive  Systems  Engineering  is  an  analysis  and  design  discipline  for  Human-System 
Integration  within  socio-technical  systems.  Analytic  methods  focus  on  the  complexities  of  work 
by  identifying  why  the  work  is  cognitively  difficult,  the  types  and  levels  of  expertise  required, 
the  functional  or  informational  structure  of  the  work  domain,  the  tasks  or  processes  employed, 
the  means  by  which  workers  develop  situation  awareness,  and  the  means  by  which  workers 
coordinate  and  communicate. 

The  specific  goal  of  Cognitive  Systems  Engineering  is  to  represent  the  work  challenges  in  such  a 
manner  as  to  inform  human-centered  design.  Analyses  typically  lead  to  forms  of  dialog  or 
representation  that  are  intended  to  support  the  design  of  process  and  technology  involved  in 
cognitive  work.  Knowledge  representation  is,  however,  a  troubling  problem.  We  have  many 
techniques  for  collecting  work-related  information  but  representation  of  that  information  is 
typically  guided  by  preference  or  intuition  rather  than  by  a  systematic  understanding  of 
representational  concepts. 

Cognitive  Work  Analysis  takes  a  more  structured  approach  to  representation  than  is  common 
within  Cognitive  Systems  Engineering.  It  is  a  multi-phase  analytic  system  that  develops 
representations  for  functional  structure,  tasks,  strategies,  collaboration  and  individual  cognitive 
performance  (Figure  1).  Although  comprehensive,  its  representations  are  nevertheless  difficult 
to  interpret  and  do  not  serve  well  as  design  artifacts,  presumably  because  they  are  predominantly 
symbolic  and  do  not  provide  evocative  distinctions  between  functions  and  processes  that  should 
be  distinguished.  A  representation  is,  in  some  sense,  intended  to  offer  a  picture,  but  for  any 
reasonably  complex  system,  its  topographical  homogeneity  defies  exploration  in  the  pursuit  of 
understanding  as,  for  example,  is  possible  with  a  representation  of  an  actual  scene  or  even  with  a 
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well-designed  map  of  a  geographical  area.  Thus,  there  is  a  general  issue  here;  how  can  Cognitive 
Systems  Engineers  represent  the  work  domain  and  activity  within  it  so  that  all  involved  in  a 
developmental  project  understand  the  domain  well  enough  to  design  solutions  for  the  cognitive 
challenges  faced  by  the  human  participants  in  the  system. 


Work  Domain 


Cognitive  Control 


Strategies 


Social  Organization 
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Figure  1:  Cognitive  Work  Analysis  is  a  multi-phase  analytic  framework  that  results  in 
design  specifications  for  functional  structure,  tasks,  strategies,  collaboration  and  individual 

cognitive  performance 


A  Systems  View 

Cognitive  Systems  Engineers  typically  focus  on  bounded  design  problems  (e.g.,  how  to  design 
an  interface,  how  to  build  a  decision  support  system  or  how  to  design  a  team  structure).  These 
types  of  interventions  are  typically  guided  by  analysis  of  a  particular  problem  area  rather  than  the 
whole  system.  Work  may  be  facilitated  locally  but  there  may  be  no  value  added  to  total  work 
output  and  there  is  even  the  possibility  that  a  local  intervention  will  negatively  impact  some 
other  aspect  of  the  system.  Cognitive  Work  Analysis  was  developed  for  the  more  extensive 
problem  of  human-centered  design  for  a  complex,  socio-technical  system.  Practitioners  of 
Cognitive  Work  Analysis  typically  gravitate  towards  trying  to  understand: 

•  The  functional  structure  of  the  system,  and 

•  How  issues  are  resolved  and  work  tasks  are  executed  within  that  functional  structure. 
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In  particular,  they  seek  to  understand  the  functional  system  organization  so  that  they  do  not 
develop  solutions  to  local  problems  that  impair  global  effectiveness. 

In  a  pioneering  effort,  Naikar,  Pearce,  Drumm,  and  Sanderson  (2003)  demonstrated  how 
Cognitive  Work  Analysis  could  be  use  to  develop  and  represent  specifications  for  a  first-of-a- 
kind  system.  The  project  reported  here  builds  upon  that  and  other  work,  focusing  on  the 
following  issues: 

•  The  distinction  between  structure  and  process, 

•  The  role  of  representation  as  a  design  artifact,  and 

•  The  integration  of  different  representations  as  a  coherent  design  artifact. 

Project  Requirements 

The  requirements  for  the  project  were  to  develop  and  demonstrate  representational  forms  for 
cognitive  processes  and  work  structures  as  follows: 

•  Develop  a  representational  form  for  work  process,  consistent  with  the  theory  of  Cognitive 
Work  Analysis,  that  can  be  linked  closely  to  the  structural  representation  derived  from 
Work  Domain  Analysis 

•  Demonstrate  how  this  representational  form  can  be  used  within  a  Command  and  Control 
domain 

•  Develop  visualization  examples,  derived  from  the  combination  of  structure  and  process 
representations,  that  depict  the  diverse  cognitively-relevant  facets  of  a  complex  socio- 
technical  system 

•  Outline  specifications  for  a  computerized  tool  that  will  allow  these  representations  to  be 
viewed  in  combinations  and  configurations  that  dynamically  illustrate  the  progress  of  a 
work  narrative  by  sequential  highlighting  of  the  elements  within  any  or  all  of  the 
representations. 

The  goal  of  this  project  is  to  develop  a  representational  form  that  can  support  exploration  of 
cognitive  demands  within  the  acquisition  process  for  a  first-of-a-kind  system. 

At  the  time  of  writing  the  proposal  for  this  work  it  had  seemed  possible  that  representations  used 
in  Systems  Engineering  might  offer  a  guide  to  development  of  more  intuitive  representations  for 
Cognitive  Systems  Engineering.  Within  this  project,  and  within  other  projects,  I  have  sought  the 
views  of  the  Systems  and  Design  Engineers  and  have  reviewed  Systems  Engineering  textbooks 
and  reports.  The  Department  of  Defense  Architectural  Framework  (DoD  Architecture 
Framework  Working  Group,  2004)  is  impressive  in  the  number  and  diversity  of  representations  it 
uses  for  systems  design.  Nevertheless,  I  have  concluded  from  my  review  of  representations  used 
within  Systems  Engineering  that  they  suffer  from  same  limitation  of  topographical  homogeneity 
as  do  the  representational  artifacts  commonly  used  within  Cognitive  Systems  Engineering.  They 
are  equally  schematic  and  are  unlikely  to  be  any  more  effective  as  artifacts — at  least  for  the 
design  of  cognitive  systems.  Thus,  it  seemed  necessary  in  this  project  to  make  a  serious  attempt 
to  advance  the  state-of-the-art  for  representation  as  a  system  design  artifact. 
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AN  INFORMATION  ENVIRONMENT  FOR  SYSTEM  DESIGN 


Information  Sources  for  System  Design 

The  general  problem  addressed  here  is  that  of  understanding  a  complex,  socio-technical  system 
sufficiently  to  redesign  its  human-system  interactions.  The  motivation  for  this  project  was 
derived  partially  from  the  inadequacy  of  current  information  sources  for  system  design.  Many 
practitioners  in  Cognitive  Systems  Engineering  and  related  disciplines  rely  on  interviews  and 
discussions  with  subject  matter  experts  and  on-site  observation  to  generate  insights.  However, 
subject  matter  experts  can  be  parochial  and  may  resist  discussions  of  the  comprehensive  view 
that  a  designer  or  developer  needs.  Observation  of  the  work  can  be  insightful  but  some  systems 
are  so  complex  that  it  is  difficult  to  get  through  the  confusion  period  to  start  assimilating 
meaningful  insights,  and  then  the  development  of  a  comprehensive  view  can  take  a  considerable 
time. 

Within  Cognitive  Work  Analysis,  domain  documents  are  used  as  one  important  source  of 
cognitive  knowledge.  Problematically,  military  documents  related  to  operational  domains 
typically  run  into  hundreds  of  pages.  Furthermore,  the  structure  and  the  abstract  nature  of  the 
information  content  for  similar  topics  is  inconsistent  between  chapters;  there  is  repetition  of 
detail  between  chapters,  and  discussions  of  critical  system  dimensions  found  in  some  chapters 
are  often  omitted  in  others.  A  more  structured,  more  consistent  and  more  comprehensive 
presentation  would  be  of  considerable  assistance  to  designers  of  systems  as  they  struggle  to 
understand  the  work  domain. 

An  Information  Environment  for  Knowledge-Based  Reasoning 

The  motivation  for  development  of  an  information  environment  to  satisfy  the  requirement  for  a 
more  structured,  more  consistent  and  more  comprehensive  presentation  is  derived  from  a 
concern  about  how  we  understand  and  reason  through  a  complex  issue  with  many  interdependent 
dimensions.  Typically,  the  dialectic  among  those  who  stand  outside  a  knowledge  domain 
revolves  first  around  one  issue  and  then  another,  often  without  clarifying  the  relationships 
between  the  two  and  without  consideration  of  other  critical  issues.  The  dialectic  can  quickly 
degenerate  into  an  agenda-driven  discussion  that  subverts  any  attempt  to  reason  about  the 
workspace.  In  contrast,  those  who  work  within  this  type  of  knowledge  domain  do  not  typically 
develop  an  appreciation  beyond  their  own  sphere,  and  may  converge  on  strategies  that  are  locally 
optimum  without  full  appreciation  of  the  global  constraints.  They  may  fall  into  comfortable 
patterns  and  may  fail  (and  even  be  reluctant)  to  explore  options,  some  of  which  could  be  more 
effective. 

The  conceptual  idea  behind  the  design  artifact,  or  reasoning  space,  developed  in  this  project  is 
that  a  cognitively  compatible  information  environment  will: 

•  Support  an  exploratory  trajectory  through  the  resources  and  functions  of  the  work  domain  at 
various  levels  of  abstraction  and  decomposition,  and 

•  Reveal  how  functions  at  one  level  of  abstraction  are  realized  by  use  of  resources  or  functions 
at  the  level  below. 

It  will  thereby  encourage  systematic  and  comprehensive  exploration  of  a  problem. 
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The  design  of  the  information  environment  developed  here  was  guided  by  the  foundational  belief 
that  effective  reasoning  relies  on  access  to  comprehensive  information  about  functional  structure 
and  about  interdependencies  within  that  structure.  Comprehensive  information  is  not,  however, 
sufficient.  That  information  must  be  organized  and  presented  so  that  those  reasoning  through  it 
can  converge  quickly  on  the  elements  of  information  that  are  central  to  the  current  discourse  and 
can  then  link  those  elements  to  construct  an  information  constellation  in  which  relationships 
between  nodes  reveal  cause,  influence,  dependency  and  effect. 

The  structure  of  this  reasoning  space  is  based  on  Rasmussen's  theory  of  problem  solving  and 
troubleshooting  (Rasmussen,  1986).  The  general  solution  prototyped  here  for  this  challenge  is  a 
pictorially-rich  reasoning  space  structured  in  a  manner  that  shows  resources  and  functions  and 
how  they  can  be  used  to  accomplish  productive  work  within  the  constraining  values  and 
priorities.  The  structure  conforms  to  Rasmussen’s  Abstraction-Decomposition  space;  a 
representation  that  reveals  interdependencies  between  information  at  different  levels  of 
abstraction  and  decomposition  in  a  representational  form  that  supports  intuitive  and  seamless 
navigation.  It  does  so  in  a  manner  that  permits  those  reasoning  about  local  issues  within  the 
space  to  explore  different  functional  configurations  that  remain  consistent  with  global  coherence. 
As  discussed  by  Rasmussen,  Pejtersen  and  Goodstein  (1994),  normal  human  problem-solving 
trajectories  map  naturally  to  the  Abstraction-Decomposition  space.  In  that  sense,  the 
Abstraction-Decomposition  space  is  a  cognitively  natural  representation  for  reasoning  about 
complex  systems. 

This  information  environment  will  support  Knowledge-Based  Reasoning  rather  than  Skill-  or 
Rule-Based  Reasoning  in  the  selected  knowledge  domain — although  it  could  serve  as  a  learning 
environment  to  help  users  develop  or  retune  their  Skill-  and  Rule-Based  Reasoning  strategies.  It 
will  be  possible,  for  example,  to  prepare  the  information  environment  so  that  it  supports  the 
Knowledge-Based  Reasoning  used  by  novices  as  they  develop  their  Skill-  or  Rule-Based 
expertise.  The  conceptual  nature  of  this  information  environment  is  general.  It  can  potentially  be 
populated  with  information  relevant  to  any  knowledge  domain. 

Issues  of  Human-Systems  Integration  in  System  Design 

Much  of  Human  Factors  deals  with  the  design  of  workstations  for  single  operators  but  Cognitive 
Systems  Engineering  is  (or  at  least  should  be)  concerned  with  design  relating  to  issues  of 
cognitive  demand  as  they  emerge  throughout  the  design  cycle.  For  complex,  distributed  work 
systems,  decisions  made  during  the  concept  refinement  and  technology  development  phases 
impact  the  effectiveness  of  the  system.  Issues  related  to  cognitive  demands  move  from  abstract 
to  concrete  and  become  more  extensive  and  detailed  as  system  development  progresses  through 
the  concept  and  technical  phases  (Figure  2). 

It  is  common  for  Human  Factors  Engineers  and  Cognitive  Systems  Engineers  to  express  concern 
about  the  failure  of  Acquisition  Managers  to  involve  them  in  resolution  of  human-systems  issues 
right  from  the  start  of  concept  refinement.  While  the  implication  of  this  complaint  is  that 
Acquisition  Managers  do  not  understand  the  significance  of  our  potential  contribution,  I  rather 
believe  that  we  have  not  had  the  tools  that  would  permit  us  to  be  effective  in  the  concept 
refinement  phase.  One  goal  of  this  project  is  to  develop  a  tool  that  can  correct  that  deficiency. 
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Figure  2:  Issues  related  to  cognitive  demand  for  collaborative,  distributed  work  become 
increasingly  concrete  and  detailed  as  systems  development  progresses  through  concept 

refinement  and  technology  development 

The  representational  products  of  Cognitive  Work  Analysis  constitute  a  structured,  consistent  and 
comprehensive  description  of  an  operational  domain  from  a  systems  perspective,  one  that  bears 
on  human-systems  issues  relevant  to  the  concept  refinement  phase  of  systems  development.  It  is 
generally  intended  that  these  products  be  developed  by  a  small  team  of  analysts  and  then  be  used 
by  other  members  of  the  design  team.  However,  the  resulting  representations  are  typically  too 
schematic  for  anyone  other  than  those  directly  involved  with  the  analysis  to  understand.  Nor  do 
they  encourage  engagement  by  subject  matter  experts  who  might  be  able  to  enrich  the  results  of 
analysis  if  they  could  understand  the  analytic  products.  Similarly,  those  who  fabricate  systems 
(e.g.,  software  engineers)  typically  ignore  the  analytic  products  of  Cognitive  Systems 
Engineering  because  they  do  not  find  them  informative.  The  purpose  of  the  information 
prototype  developed  here  is  to  demonstrate  how  it  is  possible  to  remain  consistent  with  the 
principles  of  Cognitive  Work  Analysis,  and  yet  develop  richer  and  more  meaningful 
representations  that,  despite  their  richness,  remained  succinct  enough  to  highlight  the  critical 
features  and  relationships. 

Thus,  the  principal  objective  of  this  project  is  to  demonstrate  a  style  of  integrated  representation 
for  work  structure  and  the  cognitive  processes  that  operate  within  it  that  is  more  readily 
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interpretable  by  those  who  are  not  Cognitive  Systems  Engineers,  who  have  not  participated  in 
the  analysis,  and  who  have  little  knowledge  of  the  work  domain. 

THE  FORM  AND  STRUCTURE  OF  A  REASONING  SPACE 
Information  Structure:  The  Vision 

The  general  concept  is  a  visualization  of  functional  structure  in  the  form  of  a  multilevel  view  that 
extends  the  abstraction-decomposition  depiction  into  three  dimensions  so  that  each  level  is 
represented  spatially,  with  objects  in  it  represented  evocatively  by  graphics,  icons,  pictures  or 
even  video  or  audio  clips.  The  functional  nodes  will  be  connected  via  means-ends  relations 
between  adjacent  levels  of  abstraction  and  through  decomposition  links  within  abstraction  levels. 
Because  Abstraction-Decomposition  spaces  of  large  systems  can  be  complex,  possibly 
containing  several  hundred  nodes  (Naikar  &  Sanderson,  2001),  some  of  the  features 
implemented  in  the  Work  Domain  Analysis  Workbench  developed  by  Sanderson,  Skilton, 
Cameron  and  Cao  (1998-2000)  may  be  useful.  This  Workbench  permits  selection  or  highlighting 
of  various  parts  of  the  Abstraction-Decomposition  space,  for  example  a  decomposition  cluster  or 
families  of  functions  as  connected  through  means-end  links.  These  selection  capabilities  are 
particularly  useful  for  enabling  visualization  of  various  functionally  interdependent  regions  of 
the  space. 

Information  sets  will  be  tuned  specifically  for  human-systems  design.  In  effect,  any  customized 
tuning  for  a  specific  purpose  will  omit  a  considerable  amount  of  information  about  a  work 
domain.  If  the  selection  is  well  judged,  the  excluded  information  will  not  diminish  the  practical 
value  of  the  information  environment  for  the  targeted  purpose.  Note  that  the  emphasis  here  is  on 
work  rather  than  tasks.  A  constrained  and  specific  constellation  of  information  is  typically 
needed  for  a  particular  task,  but  the  set  of  tasks  that  might  be  undertaken  in  that  workspace  (i.e., 
the  work)  will  require  a  much  broader  set  of  information.  A  comprehensive  information 
analysis,  such  as  provided  by  Cognitive  Work  Analysis,  is  essential  to  uncover  that  broader 
information  set. 

While  implementation  of  a  simulation  engine  is  beyond  the  scope  of  the  current  project,  a 
simulation  engine  will  lie  behind  the  information  structure  in  its  final  form.  That  simulation 
engine  will  be  used  to  execute  fragments  of  prototypical  tasks  (some  of  them  edge  cases).  A 
designer  will  be  able  to  monitor  scenarios  as  they  unfold  at  a  selected  speed,  and  will  be  able  to 
stop  and  reverse  the  simulation  at  will  to  capture  moments  that  can  then  be  composed  into  a 
time-line  depiction  (a  four-dimensional  visualization).  It  will  be  possible  to  execute  the 
simulation  within  any  of  the  abstraction  levels,  which  will  be  linked  through  the  means-ends 
relations  so  that  the  designer  can  track  the  task  evolution  across  levels.  Decompositions  will  also 
be  linked  so  that  the  designer  can  track  the  task  evolution  within  levels.  At  this  time,  it  is 
thought  that  an  extension  of  the  Brahms  simulation  environment  (Clancey,  Sachs,  Sierhuis,  & 
van  Hoof,  1998)  can  be  adapted  to  this  role. 

The  information  structure  has  been  developed  in  storyboard  form  in  this  report  to  show  how  a 
user  can  transition  smoothly  between  representational  layers  to  explore  a  problem.  The 
information  structure  is  as  graphical  and  iconic  as  project  resources  and  my  own  creativity  have 
allowed,  but  conversion  of  many  different  types  of  concepts  to  iconic  or  graphical  form  poses 


significant  conceptual  and  creative  challenges.  Inevitably,  at  least  some  of  the  content  is 
described  with  text. 


The  Demonstration  Concept  for  the  Reasoning  Space 

The  following  sections  of  this  report  demonstrate  a  coherent  and  economical  information  space 
for  a  constrained  knowledge  domain.  The  concept  of  a  reasoning  space  as  developed  in  this 
report  is  generic  in  the  sense  that  the  style  can  be  customized  for  any  domain  of  complex 
cognitive  work,  and  for  any  stakeholder  or  worker  who  participates  within  that  domain. 
However  the  viability  of  the  concept  needs  to  be  demonstrated  with  an  illustrative  example  for  a 
specific  knowledge  domain  and  stakeholder.  The  knowledge  domain  selected  for  illustration  is 
Time  Sensitive  Targeting.  While  it  might  be  useful  to  develop  a  knowledge  domain  for  a 
broader  problem  space,  it  is  more  important  at  this  stage  to  develop  a  concrete  and  coherent 
illustration.  That,  specifically,  was  the  goal  of  this  project.  With  an  illustration  in  place,  other 
researchers  who  grasp  the  concept  are  likely  to  have  creative  ideas  to  boost  the  technology. 

To  that  end,  the  reasoning  space  developed  here  has  been  configured  to  support  cognitive- 
systems  analysts  and  designers  who  want  to  know  about  Time  Sensitive  Targeting  in  order  to 
analyze  or  redesign  its  support  tools  or  processes.  With  some  adaptation  of  content,  this 
reasoning  space  would  be  useful  for  others,  such  as  design  engineers,  senior  air  operations 
command  staff,  or  novice  Time  Sensitive  Targeting  operators. 

In  tuning  the  information  structure  of  the  reasoning-space  for  a  specific  stakeholder,  the 
objective  is  to  provide  all  essential  but  no  superfluous  information.  In  almost  all  cases, 
information  systems  are  not  only  poorly  organized  but  also  have  considerable  information  that  is 
irrelevant  to  the  stakeholder.  Designers  of  information  systems  often  take  the  attitude  that  they 
do  not  know  what  information  stakeholders  need  and  so  they  provide  everything  they  possibly 
can.  Designers  who  develop  information  systems  of  that  type  relegate  their  responsibility  for 
identifying  the  critical  information  set  to  the  stakeholder  who  uses  it.  In  addition,  information 
systems  are  often  configured  to  satisfy  the  needs  of  many  different  types  of  stakeholders,  which 
inevitably  leads  to  much  information  that  is  irrelevant  to  specific  stakeholders.  There  is  no 
reason  in  this  day  of  reconfigurable  electronic  documents  that  information  systems  cannot  access 
a  global  information  repository,  but  be  customized  for  a  variety  of  specific  users. 

I  chose  to  tune  the  reasoning  space  for  cognitive-systems  analysts  and  designers  primarily 
because: 

•  I,  as  the  cognitive  engineer  working  on  this  project,  am  a  legitimate  stakeholder  and  thus  can 
tune  it  effectively  (to  tune  it  to  another  type  of  stakeholder  or  subject  matter  expert  would 
have  required  intensive  and  extensive  discussion  with  someone  from  that  stakeholder  or 
subject  matter  area,  which  was  beyond  the  resources  available  for  this  project). 

•  The  technical  monitor  for  this  project  is  also  a  legitimate  stakeholder,  and  will  be  able  to 
evaluate  the  product  from  that  perspective,  as  will  many  of  his  colleagues. 

It  is  content  rather  than  structure  makes  this  reasoning  space  specific  to  one  type  of  domain  and 
stakeholder.  Retuning  of  the  content  will  make  it  suitable  for  a  different  type  of  stakeholder. 


8 


THE  AIR  OPERATIONS  CENTER:  A  BENCHMARK  PROBLEM 


In  today’s  information  intensive  environment,  the  Air  Operations  Center  can  be  viewed  as  a 
benchmark  problem  for  design  of  a  complex,  socio-technical  system.  It  has  multiple  agents,  both 
people  and  information  systems,  with  considerable  need  for  coordination.  Because  the  system 
has  evolved  over  decades,  with  technological  capabilities  added  in  a  fragmented  and  piecemeal 
fashion,  the  organizational  structure  and  the  communication  and  information  support  is  not  well 
integrated.  Several  different  means  of  communication  are  available  (chat,  datalink,  secure  and 
non-secure  voice)  but  it  is  not  clear  that  these  technological  resources  are  ideal  for  the  work  they 
support.  There  is  some  movement  of  personnel  around  the  floor  of  the  Air  Operations  Center  for 
face-to-face  discussion  and  some  collaborations  are  sustained  by  individuals  pointing  things  out 
to  each  other  on  work  station  screens,  but  the  configuration  of  the  workspaces  does  not 
necessarily  support  those  sorts  of  interaction.  Large  common  screens  for  information  display  are 
available,  but  there  does  not  appear  to  be  any  coherent  understanding  of  how  they  might  usefully 
support  the  work.  Many  disparate  information  systems  are  used  to  find  information  and  post 
plans  or  decisions,  but  these  too  are  not  well  integrated. 

The  modem  Air  Operations  Center  has  evolved  into  a  system  with  1000  plus  staff  and  many 
stove-piped  information  systems.  Overstaffing  can  result  in  confusion  and  inefficiency.  There  is 
considerable  need  and  considerable  opportunity  for  redesign  based  on  the  fundamental 
constraints  of  the  work,  but  there  does  not  appear  to  be  a  coherent  concept  about  how  to  proceed. 

The  Air  Operations  Center;  Design  Goals 

The  primary  design  challenge  for  a  large,  complex  information  system  such  as  an  Air  Operations 
Center  is  the  development  of  an  efficient  and  effective  system  that  fully  integrates  the  various 
functional  components  of  distributed,  network-centric  processes.  While  much  of  the  solution  lies 
in  development  and  integration  of  technological  capability,  there  are  many  Cognitive 
Engineering  issues  that  must  be  resolved  to  establish  effective  system  performance.  Furthermore, 
in  contrast  to  the  common  approach  of  developing  technological  solutions  before  addressing 
human-system  integration  issues,  a  redesign  problem  of  this  magnitude  (and  more  generally,  any 
explicit  system  redesign  aimed  at  enterprise  transformation)  demands  resolution  of  cognitive 
issues  related  to  system  purpose  and  high-level  structure  during  concept  refinement  (Figure  2). 
Satisfactory  resolution  of  those  issues  will  ensure  an  appropriate  functional  layout  and  lead  to  an 
optimum  staffing  footprint  in  which  warfighters  with  appropriate  skills  and  appropriate  training 
are  assigned  to  fill  each  of  the  essential  positions  in  the  Air  Operations  Center. 

Once  the  functional  layout  is  determined  and  staffing  assignments  are  established,  critical 
cognitive  process  issues  become  evident;  those  such  as  how  to  support  essential 
communications,  how  to  support  critical  decisions,  how  to  help  warfighters  build  and  maintain 
situational  awareness  of  the  common  operating  picture,  how  to  avoid  process  bottlenecks  and 
performance  breakdowns  caused  by  high  workload,  and  how  to  enhance  war-fighter  performance 
with  automation  and  decision  aiding. 

In  an  organization  such  as  an  Air  Operations  Center,  resolution  of  these  cognitive  issues  starts 
with  the  nature  of  the  work.  Analyses  of  functional  structure  and  communication  patterns  are 
used  to  identify  an  optimum  physical  layout  designed  to  facilitate  collaboration  and  mission 
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accomplishment.  Appropriate  functional  allocation  will  ensure  that  the  optimum  staffing  level  is 
assigned  to  each  work  unit.  The  design  goal  is  a  staffing  level  and  organization  in  which  work 
packages  are  configured  to  ensure  acceptable  workload  levels  and  economical  communication 
overhead.  The  aim  is  a  functional  organization  of  collaborating  agents  (both  human  and 
automated)  with  the  essential  communication  links  and  appropriate  interfaces  and  decision 
support  tools.  Along  with  the  staffing  and  workload  distribution  come  training  recommendations 
to  ensure  that  the  individuals  allocated  to  each  position  have  appropriate  skills  and  expertise.  A 
comprehensive  and  well-structured  reasoning  space  has  a  large  role  to  play  in  resolution  of  these 
sorts  of  design  challenges. 


The  Use  of  the  Reasoning  Space 

Elsewhere  I  have  argued  that  modem  design  of  complex  socio-technical  systems  by  large  and 
diverse  teams  suffers  considerably  because  we  do  not  have  a  shareable  design  artifact  that  a 
design  team  can  explore  collaboratively  to  understand  the  nature  of  the  work,  the  challenges  to 
design,  and  potential  design  solutions  (Lintem,  2006).  The  reasoning  space  described  here  is 
intended  to  serve  as  such  a  design  artifact  for  one  portion  of  the  Air  Operations  Center,  the  Time 
Sensitive  Targeting  cell.  Given  sufficient  resources,  this  reasoning  space  could  be  extended  to 
cover  the  entire  Air  Operations  Center. 

There  are  two  fundamental  strategies  by  which  a  systems  designer  might  become  familiar  with 
the  complexities  of  a  large-scale  socio-technical  system: 

•  Explore  the  space  to  examine  the  available  resources  and  constraints,  and  how  those 
resources  and  constraints  can  be  accommodated  to  mission  demands,  and 

•  Work  through  scenarios  to  develop  skill  in  use  of  the  resources  and  to  explore  various 
ways  of  satisfying  mission  demands. 

Typically,  those  who  wish  to  design  technological  systems  to  be  fitted  into  an  Air  Operations 
Center  do  not  do  much  of  either. 

As  a  Cognitive  Engineer,  I  have  sought  to  execute  the  first  strategy  by  reviewing  and 
summarizing  technical  and  operational  documents.  This  can  be  both  a  time-consuming  and 
frustrating  process  because  many  documents  are  verbose,  disorganized  and  inconsistent.  In 
addition,  it  is  difficult  for  one  person  to  create  a  summary  document  that  can  then  be  used  by 
others  on  the  design  team  to  assimilate  the  essential  understandings.  I  have  sought  to  execute  the 
second  strategy  by  working  through  imagined  scenarios  as  I  examine  the  representations 
produced  from  a  cognitive  analysis  I  had  undertaken  on  the  relevant  domain.  While  I  have  found 
that  this  procedure  can  work  for  me  as  an  individual,  the  representations  I  develop  in  that  way  do 
not  appear  useful  to  others.  My  claim  is  that  a  graphical  reasoning  space,  by  revealing  the 
functional  properties  of  the  domain  and  explicating  the  interdependencies  between  levels  of 
abstraction  and  the  relationships  between  degrees  of  decomposition  in  an  evocative,  pictorial 
form,  provides  a  much  better  means  of  summarizing  that  knowledge  so  that  others  can  readily 
assimilate  it  either  by  reviewing  the  functional  properties  or  by  mentally  tracing  scenarios 
through  it. 

Those  who  use  the  reasoning  space  to  develop  a  deeper  understanding  of  the  domain  would 
examine  each  node  of  the  Abstraction-Decomposition  space  and  trace  through  its  links  to  other 
nodes,  moving  through  levels  of  either  decomposition  or  abstraction  and  then  follow  up  by 
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working  through  scenarios.  In  what  follows,  I  storyboard  both  approaches  to  demonstrate  the 
process  of  developing  a  deeper  understanding  of  Time  Sensitive  Targeting.  For  purpose  of 
illustration,  I  have  developed  a  fictitious  but  generic  scenario  and  have  then  worked  through  a 
storyboard  narrative  to  demonstrate  how  the  reasoning  space  might  be  explored  by  a  human- 
systems  designer  who  uses  the  reasoning  space  to  become  familiar  with  the  work  domain  as  a 
first  step  in  designing  human  support  systems  for  it.1 

THE  REASONING  SPACE  IN  USE 

Targeting  Scenario 

The  notional  scenario  assumes  a  military  action  by  US  and  allied  forces  in  a  country  identified  as 
Kartania.  A  forward  base  has  been  set  up  for  the  US  Air  Force  in  a  neighboring  country, 
Baranistan,  and  a  US  carrier  task  force  is  operating  off  the  coast  of  Kartania  (Figure  3).  Certain 
routine  operations  are  under  way.  An  Unmanned  Aerial  Vehicle  conducts  surveillance  from  a 
regularly  scheduled  east- west  route  along  the  northern  border  of  Kartania.  Two  air  refueling 
patterns,  one  in  central  Kartania  and  one  off  the  south  coast  of  Kartania,  are  maintained  by 
KC-10  aircraft. 

During  the  early  morning  hours,  human  intelligence  sources  have  notified  their  agency  that  the 
senior  leadership  of  a  significant  insurgent  operation  is  to  gather  during  midmoming  at  an 
identified  location.  The  human  intelligence  sources  indicate  that  the  meeting  will  commence  at 
10  a.m.  and  finish  before  noon. 

A  contingency  air  attack  plan  is  in  place  to  respond  to  this  type  of  situation.  The  goal  is  a 
precision  strike  that  will  terminate  all  members  of  the  top-level  leadership  group. 

The  Time  Sensitive  Targeting  cell  has  been  tasked  to  plan  an  air  strike  on  the  buildings  in  which 
the  meeting  will  take  place,  to  occur  after  all  the  key  insurgent  leaders  have  arrived.  The  Time 
Sensitive  Targeting  cell  is  to  identify  the  strike  aircraft  and  the  ordnance  to  be  used.  They  must 
also  schedule  fighter  air  cover  as  a  contingency  against  enemy  fighter  support  and  Electronic 
Warfare  air  platforms  to  defend  against  enemy  missile  defenses.  Air  refueling  will  be  scheduled 
as  necessary. 

The  insurgent  meeting  is  to  be  held  in  one  of  a  cluster  of  three  buildings.  The  specific  building 
to  be  used  for  the  meeting  cannot  be  identified  in  advance.  Although  surveillance  of  the  meeting 
site  is  to  be  maintained  so  that  the  arrival  of  the  insurgency  leaders  can  be  monitored  in  real¬ 
time,  the  buildings  are  connected  so  that  we  cannot  be  sure  the  meeting  will  be  conducted  in  the 
building  the  insurgents  enter.  Thus,  the  air  strike  is  to  destroy  all  three  buildings. 

The  attack  is  complicated  by  the  fact  that  this  area  is  defended  by  a  surface-to-air  missile  battery 
and  the  target  is  located  in  the  middle  of  four  sensitive  structures,  a  house  of  worship  to  the 
southeast,  a  shrine  to  the  west,  a  hospital  to  the  northeast  and  a  market  to  the  north  that  will  be 


1  Notional  data  are  used  to  support  the  storyboard  narrative,  including  aircraft  range  capabilities,  fragmentation 
footprints,  missile  acquisition  and  intercept  capabilities,  radar  capabilities,  and  jamming  capabilities. 
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filled  with  noncombatants  at  the  time  of  the  strike  (Figure  4).  The  strike  plan  must  ensure  that 
there  is  nothing  more  than  superficial  collateral  damage  and  must  also  ensure  the  safety  of  all  the 
US  strike  and  support  assets. 

A  contingency  search  and  rescue  plan  is  to  be  developed  in  case  any  air  crewmembers  have  to 
eject  from  their  aircraft. 


Representation  of  Resources  and  Constraints 

The  implication  of  Rasmussen's  theory  of  problem  solving  (Rasmussen,  1 986)  is  that  problem 
solvers  navigate  opportunistically  through  an  Abstraction-Decomposition  space.  They  inspect 
resources  and  functional  capabilities  to  develop  a  strategy  that  brings  functional  capabilities  into 
line  with  the  purpose  and  values  that  act  as  constraints  on  action.  There  is  no  optimum  starting 
point  in  the  Abstraction-Decomposition  space.  The  prime  requirement  is  that  the  trajectory 
through  the  space  visits  the  functional  nodes  that  help  the  problem  solver  understand  the 
situation  and  develop  a  plan  that  will  satisfy  the  work  demands.  Targeting  should  conform  to 
this  pattern  of  behavior  since  planning  is  an  exercise  in  developing  a  solution  to  a  problem. 


Figure  3:  Area  of  operations  for  the  notional  targeting  scenario 

The  narrative  of  the  following  paragraphs  is  fictional,  created  for  the  purposes  of  illustration.  It 
is  a  description  of  a  trajectory  a  human-systems  designer  might  follow  in  seeking  to  become 
informed  about  the  nature  of  the  work  of  Time  Sensitive  Targeting  prior  to  embarking  on  design 
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of  a  support  tool  for  this  work  domain.  In  practice,  that  human-systems  designer  would 
preferably  confirm  his  or  her  newfound  understanding  by  subsequently  working  though  the  space 
with  a  subject  matter  expert,  and  then  might  engage  with  others  on  the  design  team  to  again  work 
through  the  reasoning  space  to  help  them  generate  a  useful  level  of  understanding  about  the  work 
domain.  In  particular,  the  reasoning  space  is  intended  to  be  an  exploratory  and  collaborative 
environment  in  which  colleagues  can  work  through  problems  and  develop  ideas  in  a  manner  that 
is  not  possible  with  a  normal  document  or  even  with  a  graphically  rich  presentation. 

In  addition,  readers  should  recall  that  this  reasoning  space  has  been  customized  specifically  for 
the  use  of  a  human-systems  designer  rather  than  Air  Operations  staff.  It  should  be  remembered, 
however,  that  the  concept  of  a  reasoning  space  is  intended  to  be  generic.  Thus,  this  Time 
Sensitive  Targeting  reasoning  space  could  be  customized  for  those  who  are  to  work  as  Air 
Operations  staff  but  are,  as  yet,  novices  in  this  work  domain,  or  it  could  even  be  customized  for 
experts  in  this  work  domain  who  wish  to  explore  new  ways  of  doing  things  or  strategies  for 
dealing  with  difficult  challenges. 
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Figure  4:  The  target  is  a  defended  cluster  of  three  buildings  surrounded  by  sensitive 

structures 
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The  Human-Systems  Designer’s  Exploration  of  the  Reasoning  Space 

The  reasoning  trajectory  might  be  initiated  by  review  of  System  Mission  (Figure  5).  The  human- 
systems  designer  might  examine  the  relevant  node  and  then  follow  the  means-ends  links  to  the 
level  of  Operational  Principles  &  Values  to  become  familiar  with  the  constraining  values.  These 
are  depicted  as  a  pair  of  polar  stars  calibrated  in  terms  of  conformance  to  military  doctrine  (left) 
and  guidance  abstracted  from  various  operational  documents  (right).  The  labels  for  each  radial 
are  intended  as  summary  reminders  but  will  not  be  particularly  meaningful  to  the  novice. 
Interrogation  of  a  particular  label  will  activate  an  embedded  hyperlink  leading  to  a  succinct 
description  (here  shown  as  a  call  out,  of  which  three  are  shown  in  Figure  5)  of  what  that  label 
represents  (summarized  from  Air  Force  Manual  1-1,  Basic  Aerospace  Doctrine  of  the  United 
States  Air  Force).  An  even  more  detailed  (but  nevertheless  still  succinct)  explanation  may  lie 
behind  that  summary  statement. 

The  human-systems  designer  might  then  move  to  a  review  of  the  General  Mission  Functions.  In 
a  previous  analysis  (Lintem,  2005),  I  identified  General  Mission  Functions  as  shown  in  Figure  6. 
To  maintain  global  awareness,  it  was  thought  important  to  locate  the  targeting  function  within 
the  overall  “kill  chain”  that  describes  the  general  mission  process  (as  illustrated  in  Figure  6).  As 
noted  in  the  figure,  targeting  is  the  only  kill-chain  function  that  is  the  responsibility  of  the  Time 
Sensitive  Targeting  cell. 

During  development  of  this  reasoning  space,  it  became  apparent  that  there  is  a  characteristic 
(although,  flexible)  sequence  to  development  of  a  targeting  plan.  That  sequence,  laid  out  in 
Figure  7,  illustrates  the  strategy  of  first  identifying  the  appropriate  assets  (with  consideration  to 
the  nature  of  the  target),  of  then  ensuring  that  the  spatial  constraints  can  be  satisfied,  and  finally 
ensuring  that  the  temporal  constraints  can  be  satisfied.  Figure  7  also  shows  the  relationship  of 
the  previously  identified  mission  functions  to  each  of  these  stages.  To  understand  the  process  of 
plan  development,  the  human-systems  designer  might  navigate  up  and  down  the  hierarchy  and 
visit  appropriate  levels  of  decomposition  to  explore  the  way  in  which  each  of  these  phases  is 
resolved. 

It  may  then  be  useful  to  examine  the  Physical  Resources  &  Constraints  nodes  to  overview  what 
is  available  and  where  challenges  may  arise  (Figure  8).  As  the  human-systems  designer  checks 
the  Physical  Resources  &  Constraints,  he  or  she  would  be  able  to  interrogate  embedded 
hyperlinks  to  access  more  detailed  descriptions  and  at  other  times  follow  means-ends  links  to  the 
Technical  Functions  &  Contextual  Effects  (Figure  9)2 3  to  identify  the  functional  capabilities  or 
functional  constraints  generated  by  physical  resources  or  features.  The  systems  analyst  might 
first  examine  the  assets  (both  allied  and  adversary)  that  can  be  involved  in  an  air  attack  mission 
at  the  physical  resource  level  (parameters  of  number,  location  and  availability)  and  then  follow 


2  The  abstraction  labels  “System  Mission,”  “Operational  Principles  &  Values,”  “General  Mission  Functions,” 
“Technical  Functions  &  Contextual  Effects,”  and  “Physical  Resources  &  Constraints”  referred  to  below  correspond 
respectively  to  the  labels  “System  Purpose,”  “Values  &  Priorities,”  Purpose-Related  Functions,”  “Physical 
Functions  &  Effects,”  and  “Physical  Properties”  as  defined  in  Appendices  A  and  B. 

3  The  iconic  representations  in  Figure  9  are  discussed  in  the  text  accompanying  Figure  10  through  Figure  17. 
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Figure  5:  Purposes  and  Operational  Principles  &  Values  with  callouts  that  show  more 

detailed  descriptions  of  selected  labels 
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Figure  7:  General  Mission  Functions  showing  a  characteristic 
Capability->Spatial->Temporal  planning  sequence 
(solid  arrows  map  the  characteristic  sequence,  and  the  dashed 
arrows  point  to  kill  chain  elements  that  participate  in  each  stage) 

During  exploration  at  the  level  of  Technical  Functions  &  Contextual  Effects  (Figure  9),  it  would 
be  necessary  to  examine  capabilities  of  different  technical  systems  and  effects  of  different 
environmental  systems  in  detail.  It  might  be  useful  to  start  with  aircraft  capabilities.  Figure  10 
depicts  aircraft  range  envelopes.  Plan  development  should  take  account  of  refueling  possibilities 
and  effects  on  aircraft  performance  of  winds  at  cruise  altitude.  Depictions  of  other  aircraft 
functional  properties  have  not  yet  been  developed  but  samples  of  those  that  are  probably 
important  are  indicated  in  the  callouts.  The  two  callouts  are  distinguished  between  structural  and 
organizational  properties.  Interviews  with  subject  matter  experts  in  a  previous  project  indicated 
that  datalink  (not  currently  available  on  all  aircraft),  identified  in  Figure  10  as  Target  Data 
Reception,  offers  an  important  support  function.  The  voice  transmission  of  target  coordinates  to 
aircraft  is  time-consuming,  cognitively  effortful,  and  error-prone.  A  functional  capability  to 
accept  target  coordinates  via  some  form  of  upload  reduces  workload  for  both  the  targeteer  and 
the  pilot  and  reduces  potential  for  error. 

It  would  also  be  useful  to  examine  the  capabilities  of  the  adversary’s  air  assets,  especially  fighter 
aircraft  that  might  be  directed  to  intercept  allied  air  assets.  It  would  be  important  to  become 
familiar  with  those  enemy  capabilities  in  relation  to  the  capabilities  of  allied  defensive  and 
offensive  systems.  How,  for  example,  do  enemy  fighter  aircraft  rate  in  relation  to  allied  fighter 
aircraft  in  terms  of  armaments,  maneuverability,  and  enemy  detection  and  targeting  capability? 
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Figure  8:  A  layout  of  Physical  Resources  and  Constraints 
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Figure  9:  A  layout  of  Technical  Functions  &  Contextual  Effects  (see  Figure  10-Figure  17  for  clarification  of  illegible  text) 


Figure  10:  Left  panel  depicts  notional  aircraft  range  envelope  (maximum  range  including 
return  without  refueling)  and  identifies  other  important  aircraft  properties  for  which 
graphical  form  have  not  yet  been  developed;  right  panel  contrasts  notional  maximum 

range  to  target  with  or  without  refueling 


Ordnance  capabilities  are  also  important.  The  human-systems  designer  would  examine 
weapons-effect  footprints  at  the  Technical  Functions  &  Contextual  Effects  level.  Figure  1 1 
shows  a  fragmentation  footprint  for  a  30°  versus  60°  delivery  angle-of-attack  relative  to  the 
impact  point,  and  distinguishes  between  ordnance  intended  for  surface  structures  versus  buried, 
reinforced  structures.  Other  types  of  capabilities,  not  yet  depicted  in  this  reasoning  space,  would 
identify  anti-personnel  ordnance  and  weapons  suitable  for  damaging  aircraft  runways  or 
reinforced  surface  structures. 

The  human-systems  designer  might  then  examine  the  challenges  posed  to  attacking  aircraft  by 
defensive  surface-to-air  missiles.  The  left  panel  of  Figure  12  depicts  the  probability  that  a 
missile  will  intercept  an  incoming  aircraft  based  on  offset  of  the  aircraft  as  it  passes  the  missile 
battery.  The  right  panel  of  Figure  12  depicts  the  risk  of  aircraft  loss  from  a  missile  strike  based 
on  the  missile  velocity  at  intercept  and  the  efficiency  of  electronic  counter  measures  on  the 
aircraft.  Missile  velocity  fades  rapidly  with  distance  traveled  (possibly  by  two-thirds  over  its 
effective  range,  as  shown  in  Figure  13)  once  a  missile  reaches  its  maximum  velocity, 
significantly  reducing  the  probability  first  of  an  intercept  and  then  of  a  strike  with  increase  in 
distance  traveled. 
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Figure  11:  Notional  ordnance  fragmentation  footprint  for  two  delivery  angles  (with  a 
suitable  modeling  tool,  ordnance  designation  and  weight  would  adjust  the  footprint) 
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Figure  12:  Notional  probability  of  intercept  of  a  target  by  a  surface-to-air  missile  based  on 
range  and  offset  to  incoming  aircraft  (left  panel)  and  notional  risk  of  aircraft  loss  from  a 
missile  strike  based  on  missile  velocity  at  intercept  and  efficiency  of  electronic  counter 

measures  (right  panel) 
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Figure  13:  Illustrative  velocity  profile  for  surface-to-air  missile  showing  speed  decay  versus 

distance  traveled 


Suppression  of  enemy  air  defense  (SEAD)  capabilities  is  also  relevant  to  consideration  of  enemy 
defense  capabilities.  Figure  14  depicts  both  the  radar  tracking  capability  for  a  surface-to-air 
missile  (SAM)  and  a  jamming  capability  for  an  aircraft  with  enemy  defense  suppression 
capabilities.  The  primary  lobe  of  the  missile  tracking  radar  is  the  one  of  most  concern. 
Suppressive  radar  jamming  is  most  effective  along  the  axis  of  that  lobe.  Jamming  effectiveness 
reduces  with  distance  of  the  jammer  from  the  missile  radar  and  with  offset  from  its  primary  lobe. 
The  99%  jamming  effectiveness  threshold  is  depicted  (Figure  14)  as  a  tradeoff  between  distance 
and  offset.  The  range  of  the  anti-radiation  missiles  that  the  aircraft  can  deliver  to  home  in  on  the 
surface-to-air  missile  radar  is  also  depicted.  As  shown  here,  it  is  desirable  that  the  defense 
suppression  aircraft  stand  outside  the  primary  acquisition  range  of  the  missile.  As  depicted  in 
Figure  14,  a  30°  jamming  cone  from  that  range  lies  within  the  99%  jamming  effectiveness 
threshold  to  reveal  that  effective  jamming  is  possible  from  that  distance  if  the  standard  30° 
jamming  cone  is  used. 

Most  missions  will  require  air  refueling.  Tanker  resources  are  typically  in  demand  and  so  the 
development  of  a  refueling  schedule  for  a  mission  with  many  extra  aircraft  can  be  demanding 
and  time-consuming.  The  graphic  of  Figure  15  shows  a  12-hour  tanker  timeline  in  which  20- 
minute  slots  are  shown  as  available,  allocated,  priority  allocated,  or  unavailable.  While  any 
allocated  slot  can  be  reallocated  to  a  higher  priority  requirement,  a  priority  icon  attached  to  an 
allocated  slot  offers  a  caution  that  this  slot  has  already  been  allocated  to  a  high  priority 
requirement.  Unavailable  slots  indicate  the  times  that  a  tanker  is  not  on  its  refueling  station. 
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Figure  14:  A  notional  comparison  of  target  acquisition  range  for  a  surface-to-air  missile 
and  the  target  detection  lobe  of  its  radar  versus  an  air  borne  electronic  suppression 
capability  and  range  of  an  air  delivered  anti-radiation  missile 
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Figure  15:  A  notional  tanker  scheduling  timeline 
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Figure  16  depicts  the  capabilities  of  three  types  of  airborne  surveillance  sensors  in  terms  of  their 
effectiveness  in  different  visibility  conditions,  the  size  of  objects  they  can  resolve  and  whether  or 
not  they  can  detect  motion.  Figure  17  shows  the  surveillance  range  and  footprint  as  impacted  by 
the  subtended  angle  of  the  surveillance  cone  and  the  lookdown  angle.  The  upper  panel  in  this 
figure  shows  a  surveillance  footprint  within  the  effective  surveillance  range.  In  the  lower  panel, 
the  footprint  extends  beyond  the  effective  surveillance  range,  suggesting  that  some  adjustment  in 
the  surveillance  parameters  is  desirable.  Possibly  there  would  be  some  benefit  in  reducing 
subtended  angle  of  coverage  to  enhance  resolution  or  it  may  be  desirable  to  decrease  standoff 
range  to  ensure  that  the  entire  footprint  is  within  the  effective  surveillance  range. 
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Figure  16:  Functional  capabilities  of  different  surveillance  systems  in  terms  of  detection 
capability  under  different  visibility  conditions,  resolution  capability  for  different  sized 

objects  and  motion  detection  capability 
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Figure  17:  Surveillance  cone  for  airborne  system  illustrating  range  and  footprint  as 
impacted  by  the  subtended  angle  of  the  surveillance  cone  and  the  lookdown  angle  (upper 
panel  shows  footprint  within  range;  lower  panel  show  footprint  extending  beyond  range) 
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Once  the  human-systems  designer  has  reviewed  the  functional  capabilities  of  technical  systems, 
he  or  she  might  review  general  weather  patterns  that  could  impact  operations.  Weather 
forecasting  is  one  of  the  few  areas  relevant  to  this  domain  in  which  there  has  been  considerable 
progress  in  developing  innovative  and  evocative  visualizations.  Because  the  review  of  that 
material  would  require  a  major  effort  and  was  not  the  primary  goal  for  this  project,  an  effective 
visualization  of  weather  and  its  effects  have  been  left  for  the  future. 


The  human-systems  designer  might  then  examine  priority  target  types  and  also  commercial, 
industrial,  social  and  cultural  structures  that  could  potentially  influence  a  targeting  plan.  The  left 
panel  of  Figure  18  shows  icons  that  could  be  used  to  represent  high-priority  targets.  The  middle 
panel  has  icons  for  commercial  and  industrial  infrastructure;  systems  that  under  some  scenarios 
will  be  viewed  as  targets  and  under  other  scenarios  will  be  viewed  as  sensitive  structures  that 
should  not  be  damaged.  The  right  panel  has  icons  for  cultural  and  religious  structures  that 
should  not  be  damaged  under  any  scenario. 


The  human-systems  designer  should  not  yet  need  to  review  specific  numbers  and  locations  of 
objects  of  different  types  (or  specific  weather  forecasts  for  particular  periods)  but  should  become 
familiar  with  the  types  of  issues  that  might  be  encountered  during  planning.  Particularly  with 
generic  target  types,  he  or  she  might  consult  linked  documents  to  assess  special  characteristics 
that  may  be  of  interest  in  terms  of  their  vulnerability  to  different  types  of  weapons,  preferred 
targeting  locations,  and  expected  or  desired  effects  of  a  strike  against  them. 
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Figure  18:  Priority  target  types  and  also  commercial,  industrial,  social  and  cultural, 
structures  that  could  potentially  be  designated  as  sensitive  and  to  be  protected  against 

collateral  damage 

It  may  be  useful,  for  example,  for  the  human-systems  designer  to  be  aware  that  the  type  of 
structure  and  the  constituent  materials  of  a  bridge  to  be  targeted  for  destruction.  This  can  impact 
the  targeting  plan  because  they  determine  the  bridge’s  vulnerability  to  damage  or  destruction. 
Challenges  confronting  replacement  of  that  bridge  can  also  impact  targeting  decisions  in  the 
event  that  our  own  forces  might  want  to  use  it  in  the  near  future,  or  if  its  reconstruction  would  be 
part  of  a  future  rebuilding  program.  A  desire  to  temporarily  disable  rather  than  destroy  a  bridge 
would  impact  the  choice  of  ordnance. 

The  functional  constraints  imposed  on  targeting  by  some  physical  features  such  as  houses  of 
worship,  shrines,  or  community  centers  may  be  self-evident  from  the  physical  appearance  or 
their  identifying  title.  It  may  seem  overly  pedantic  to  identify  functions  of  objects  such  as  a 
hospital  when  they  seem  so  obviously  implied  by  its  name,  but  the  functions  of  many  other 
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physical  features  are  obscure  and  it  is  essential  within  this  reasoning  structure  to  maintain 
consistency  within  and  across  levels  of  abstraction.  In  particular,  the  inclusion  of  obvious 
relationships  provides  intuitive  illustrations  of  how  to  use  the  reasoning  space,  while  omission  of 
those  obvious  relationships  can  potentially  generate  confusion  about  how  it  works. 

Concerns  about  collateral  damage  to  sensitive  features  would  flow  from  statements  in  guidance 
documents  about  protection  of  non-combatants,  cultural  sensitivities,  and  adherence  to 
international  law.  Many  of  these  concerns  may  also  seem  obvious  (for  example,  is  it  necessary 
to  point  out  that  we  should  not  indiscriminately  place  non-combatants  at  risk?)  but  consistency 
within  and  across  levels  of  abstraction  is  crucial  at  all  levels.  Furthermore,  even  self-evident 
concerns  can  diminish  in  significance  during  the  heat  of  combat. 

At  this  stage,  the  human-systems  designer  would  know  a  good  deal  about  the  resources  and 
constraints  related  to  planning  for  time  sensitive  targeting.  To  enrich  and  extend  his  or  her 
knowledge  and  to  make  it  more  robust,  it  would  now  be  useful  to  embark  on  a  planning  exercise. 

A  PLANNING  TRAJECTORY  FOR  A  HUMAN-SYSTEMS  DESIGNER 

This  section  illustrates  how  a  human-systems  designer  or  analyst  might  use  a  scenario  and  the 
reasoning  space  to  determine  the  nature  of  the  work  conducted  by  planners  in  a  Time  Sensitive 
Targeting  cell.  Possibilities  for  tuning  the  reasoning  space  for  those  who  are  to  work  as  Air 
Operations  staff  should  also  become  clearer  as  one  reads  through  the  exercise  illustrated  here. 

The  exercise  to  be  described  is  aimed  at  development  of  a  targeting  plan  for  the  scenario 
described  in  the  earlier  Targeting  Scenario  section  of  this  report.  The  human-systems  designer 
or  analyst  acts  as  a  planner  within  a  Time  Sensitive  Targeting  cell.  He  or  she  starts  by 
examining  the  general  layout  of  the  area  of  operations  (Figure  3),  the  general  topography,  the 
locations  and  general  layout  of  cities,  and  the  locations  and  layout  of  major  industrial, 
commercial  and  public  infrastructure.  He  or  she  would  then  commence  to  work  through  a 
process  similar  to  that  undertaken  by  a  planner  within  a  Time  Sensitive  Targeting  cell. 

Functional  Coordination 

Military  resources  have  functional  capabilities  and  part  of  the  planning  challenge  tackled  by  a 
targeteer  is  to  ensure  that  the  allied  capabilities  committed  to  the  mission  have  the  functionality 
required  not  only  to  fulfill  their  role  in  the  mission  but  also  to  counter  any  functional  capabilities 
the  adversary  might  use  to  disrupt  the  mission.  Nonmilitary  features,  on  the  other  hand,  can  be 
either  resources  (e.g.,  distinctive  features  can  be  used  to  guide  a  strike  pilot  to  a  target)  or 
constraints  (e.g.,  weather  effects  can  restrict  options  or  sensitive  structures  can  limit  the  nature  of 
a  strike). 

Once  a  target  is  identified,  delivery  options  would  be  reviewed  and  a  selection  made.  Guided 
missiles  or  high-level  bombers  are  possibilities,  but  for  this  exercise  accuracy  is  deemed 
important,  so  a  piloted  aircraft  that  will  permit  visual  sighting  of  the  target  area  during  the  attack 
run  is  required.  Because  the  target  area  lies  within  potentially  hostile  territory,  a  high-speed, 
long-range  platform  with  the  ability  to  outmaneuver  enemy  aircraft  and  to  counter  surface-to-air 
missiles  will  be  used.  While  the  A- 10  Thunderbolt  can  carry  suitable  ordnance,  its  performance 
profile  is  unsuitable  for  this  mission.  A  B-1B  Lancer  would  be  effective  but  to  deploy  it  for  an 
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isolated  target  would  be  an  inefficient  use  of  a  scarce  resource,  as  would  the  use  of  an  F-l  17A 
Nighthawk.  Both  the  F-15E  Strike  Eagle  and  the  F-l  8  Hornet  have  suitable  performance  profiles 
and  can  carry  suitable  ordnance.  The  decision  is  made  to  schedule  one  of  those  aircraft  types  for 
the  mission.4 

To  select  a  suitable  weapon,  the  human-systems  designer,  acting  as  a  planner,  would  locate  and 
identify  the  target,  examine  its  physical  characteristics  and  consider  the  desired  effect  of  the 
strike  against  it.  The  target  is  a  non-reinforced  surface  structure  that  is  to  be  demolished.  The 
destructive  effect  of  the  available  ordnance  would  be  assessed  via  the  depiction  of  fragmentation 
footprints  (Figure  1 1)  at  the  level  of  Technical  Functions  &  Contextual  Effects  to  confirm  this 
selection.  That  fragmentation  footprint,  when  laid  over  the  target  area  at  the  General  Mission 
Function  level,  suggests  that  a  2,000-pound  warhead  would  offer  suitable  destructive  capability 
(Figure  19,  left  panel).  This  leads  to  selection  of  a  2,000-pound  MK-84  general-purpose  bomb. 


Figure  19:  The  target  area  overlaid  with  an  ordnance  fragmentation  footprint  for  two 
delivery  angles  with  left  panel  showing  unacceptable  risk  of  collateral  damage  which  is 
corrected  in  the  right  panel  by  adjustment  of  ordnance  impact  point  and  selection  of  a 

30-degree  delivery  angle 

Previous  work  (Lintem,  2006)  has  shown  that  a  planner  will  often  assess  issues  related  to 
collateral  damage  early  in  planning  by  examining  the  physical  layout  of  the  space  surrounding 
the  target  to  identify  sensitive  structures  or  social  environments  that  must  be  protected.  In  this 
case,  the  human-systems  designer  acting  as  a  planner  ascertains  that  the  targeted  area  is 
surrounded  by  several  sensitive  structures  (Figure  4).  The  functions  of  these  structures  are 
obvious,  but  could  be  confirmed  by  tracing  the  means-ends  links  between  the  levels  of  Physical 
Resources  &  Constraints  and  Technical  Functions  &  Contextual  Effects.  The  fact  that  these  are 
sensitive  structures  that  should  be  protected  from  collateral  damage  is  also  obvious,  but  could  be 
confirmed  by  tracing  the  means-ends  links  through  the  General  Mission  Functions  to  the  level  of 


4  For  reasons  discussed  later,  access  to  further  details  of  aircraft  resources  is  described  in  a  later  section  of  this 
report,  headed  Resource  Descriptions  via  Embedded  Hyperlinks 
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Operational  Principles  &  Values  to  access  relevant  instructions,  guidance,  and  rules  of 
engagement. 

The  potential  for  unacceptable  collateral  damage  requires  further  thought  about  the  destructive 
effects  of  the  ordnance  and  its  fragmentation  footprint.  An  approach  heading  of  050°  (to 
approach  the  target  from  230°)  is  selected  initially,  but  it  is  apparent  from  inspection  of  Figure 
19  (left  panel)  that  the  fragmentation  footprint  of  the  selected  ordnance  overlaps  some  of  the 
sensitive  surrounding  structures.  One  option  might  be  the  evaluation  of  whether  less  destructive 
ordnance  would  damage  the  target  sufficiently.  However  for  this  scenario  illustration,  the  option 
pursued  is  reassessment  of  the  weapon  delivery  direction  and  angle  of  attack,  which  also 
influences  the  fragmentation  footprint.  Two  fragmentation  footprints  are  shown,  one  based  on  a 
30°  angle  of  delivery  and  the  other  on  a  60°  angle  of  delivery.  Both  delivery  angles  show  an 
unacceptable  risk  of  collateral  damage,  the  fragmentation  footprint  for  the  30°  angle  overlapping 
the  hospital  and  at  that  for  the  60°  angle  overlapping  the  market. 

Although  the  most  desirable  impact  point  from  a  target  destruction  point  of  view  is  in  the  center 
of  the  cluster  of  three  buildings,  total  target  destruction  can  be  achieved  even  if  it  is  displaced 
slightly.  The  adjustment  shown  in  Figure  19  (right  panel)  clears  both  footprints  from  the  market 
and  the  hospital.  However,  the  footprint  for  the  60°  delivery  does  not  cover  the  targeted 
structures  entirely  and  now  impinges  on  another  sensitive  structure,  the  shrine.  Thus  a  30°  angle 
of  delivery  is  selected  for  the  adjusted  impact  point. 

The  fit  is  tight  however,  so  reliability  and  accuracy  are  imperative.  Thus  a  guided  weapon  is 
required.  Both  the  F-15E  Strike  Eagle  and  the  F-18  Hornet  can  deliver  bombs  guided  by  the 
Global  Positioning  System  or  by  laser.  The  target  itself  is  easily  distinguished  from  the 
surround,  so  laser-guidance  (as  provided  by  the  Guided  Bomb  Unit-24,  which  can  be  fitted  to  the 
MK-84  general-purpose  bomb  as  shown  in  Figure  8)  is  preferred  over  Global  Positioning  System 
guidance  because  the  pilot  will  be  able  to  ensure  visually  that  the  weapon  is  aimed  at  the  right 
structure.  The  pilot  will  need  a  clear  and  unambiguous  description  of  the  target  and  of 
distinguishing  landmarks  for  visual  tracking  onto  the  target.  Landmarks  that  can  guide  the  pilot 
to  the  target  should  be  sufficiently  visible  so  that  pilot  workload  is  maintained  within  a 
reasonable  level. 

The  use  of  laser-guidance  is  further  encouraged  by  the  forecast  for  the  critical  period,  which  is 
for  excellent  air-to-ground  visibility.  Any  possibility  of  extensive  cloud  cover  would  require  an 
adjustment  of  this  plan  to  guidance  by  Global  Positioning.  Because  dense  cloud  cover  is 
common  in  the  attack  area  for  this  time  of  year  (as  assessed  by  examination  of  weather  effects  at 
the  level  of  Technical  Functions  &  Contextual  Effects),  there  is  some  concern  about  reliance  on 
laser  guidance.  This  issue  is  resolved  by  ensuring  that  two  attack  aircraft  that  carry  bombs 
guided  by  the  Global  Positioning  System  (Figure  8  shows  that  a  MK-84  general-purpose  bomb 
can  be  fitted  with  a  Guided  Bomb  Unit-32)  are  in  the  vicinity,  ready  to  be  tasked  for  the  attack  at 
the  last  moment.  Aircraft  with  data-link  capability  are  preferred  for  this  backup  role  to  avoid  the 
need  for  the  time-consuming  and  error-prone  procedure  of  transmitting  target  coordinates  by 
voice  link. 

The  analyst  acting  as  a  planner  should  notice  that  the  target  (in  addition  to  being  nestled  within 
sensitive  structures)  is  located  near  a  defensive  missile  site  and  the  selected  approach  to  the 
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target  passes  near  it.  He  or  she  might  reconsider  the  direction  of  approach  but  cannot  avoid 
significant  collateral  damage  from  any  other  approach  course  except  the  reciprocal  approach 
heading  of  230°,  which  only  transfers  the  problem  from  target  approach  to  target  egress.  (A 
decision  about  whether  one  is  more  desirable  than  the  other  is  the  sort  of  issue  that  should  be 
easily  resolved  within  this  reasoning  space  but,  as  with  many  other  relevant  issues,  the 
appropriate  subject  matter  expertise  has  not  yet  been  acquired.) 

A  depiction  of  the  hostile  surface-to-air  missile  capability  at  the  Mission  Function  level  reveals  a 
potential  danger  for  allied  strike  aircraft  (Figure  20).  The  proximity  of  the  target  area  (and 
especially  the  F-15  attack  course)  to  a  hostile  surface-to-air  missile  site  suggests  the  need  for 
suppression  of  enemy  air  defense  capability.  This  functional  capability  can  be  found  at  the 
Physical  Function  level  and  means-ends  links  can  be  followed  from  there  to  the  Physical 
Resource  level  to  identify  the  aircraft  types  that  can  provide  this  functional  capability.  The 
detection  lobe  of  the  missile  radar  and  the  missile  range  should  be  examined  at  the  Physical 
Function  level  (Figure  14)  and  then  compared  at  the  Mission  Function  level  (Figure  20)  to  the 
suppression  capabilities  of  the  selected  Electronic  Warfare  aircraft  when  standing  off  outside  the 
surface-to-air  missile  range. 


Figure  20:  F-15  attack  course  in  comparison  to  SAM  kill  zone,  SAM  radar  primary  lobe 

and  SEAD  suppression  lobe 

An  EA-6B  Prowler  is  selected  as  an  appropriate  aircraft  currently  operating  in  the  theater  that 
would  be  suitable.  This  aircraft  carries  anti-radiation  missiles  that  can  home  in  on  the  surface-to- 
air  missile  radar.  However,  because  the  range  of  that  anti-radiation  missile  lies  within  the 
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primary  acquisition  range  of  the  surface-to-air  missile,  the  use  of  the  anti-radiation  missile  in  this 
circumstance  is  not  recommended.  The  Prowler  will  be  assigned  to  a  station  beyond  the  surface- 
to-air  missile  range,  but  located  along  the  line  of  the  attack  to  ensure  optimum  jamming  as  the 
radar  main  lobe  is  directed  at  the  attacking  aircraft  (Figure  20). 


The  adversary  has  a  small  Air  Force  with  some  fighter  aircraft  that  could  disrupt  this  mission. 

To  guard  against  that  possibility,  the  mission  plan  will  include  the  scheduling  of  fighter  cover  for 
all  other  aircraft  involved  in  the  mission.  It  is  known  that  the  adversary  has  several  Russian 
SU-27s,  which  are  thought  to  have  an  edge  in  maneuverability  over  the  F-15  (a  suitable  depiction 
of  this  comparison  has  not  yet  been  developed).  Because  of  that,  the  F-18  Hornet  will  be 
preferred  for  this  role. 


The  plan  calls  for  continued  surveillance  of  the  insurgency  meeting  site.  The  human  intelligence 
assets  that  first  learned  of  the  meeting  will  continue  to  observe  the  site  and  report  back  when 
possible,  but  there  is  a  need  for  assets  that  can  report  back  in  real  time  as  they  monitor  the  site. 
The  surveillance  function  at  the  Physical  Function  level  is  linked  to  suitable  surveillance  assets  at 
the  Physical  Resource  level.  The  Global  Hawk  offers  suitable  capability.  It  has  an  optical 
surveillance  capability  to  permit  visual  identification  in  good  weather  and  a  Synthetic  Aperture 
Radar  and  infrared  capability  to  infer  arrival  and  departure  of  vehicles  if  the  optical  sensing 
system  is  obscured  by  unfavorable  weather  conditions.  A  Global  Hawk  will  be  scheduled  to 
maintain  the  target  area  under  constant  surveillance  (Figure  21). 


Figure  21:  Critical  mission  functions 
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Search  and  rescue  assets  will  be  scheduled  to  wait  at  an  appropriate  location  to  rescue  airmen  in 
case  an  aircraft  is  lost.  The  HH-60G  Pave  Hawk  is  identified  as  the  appropriate  asset  for  this 
task.  Its  equipment  includes  a  personnel  locating  system  compatible  with  the  PRC-1 12  survival 
radio  carried  by  airmen  that  provides  range  and  bearing  information  to  a  survivor's  location.  It 
has  a  hoist  capable  of  retrieving  two  to  three  airmen  from  a  hover  height  of  200  feet,  thereby 
obviating  the  necessity  to  land  during  a  rescue  mission.  It  carries  surface-to-air  countermeasures 
that  could  be  needed  in  this  scenario  environment.  Finally,  it  has  an  in-flight  refueling  capability 
that  may  be  needed  if  the  search  and  recovery  mission  extends  over  a  period  of  time,  and  it  has 
light  offensive  capability  in  the  form  of  7.62-mm  machine  guns  that  could  be  brought  to  bear 
against  ground  troops  during  a  recovery. 

Spatial  Coordination 

It  is  now  necessary  to  identify  specific  assets  for  the  coordinated  strike  (tail  numbers).  Once 
aircraft  types  have  been  identified,  their  hyperlinks  can  be  interrogated  to  bring  up  documents 
that  identify  their  home  locations,  their  tail  numbers,  and  their  availability  for  this  mission. 

For  this  scenario,  suitable  air  assets  are  available  in  both  the  carrier  task  force  off  the  south  coast 
of  Kartania  and  the  USAF  forward  air  base  in  Baranistan  (Figure  21),  but  the  carrier  task  force  is 
closer  to  the  target  area  and  its  resources  are  preferred.  Nevertheless,  interrogation  of  the  air 
asset  hyperlinks  reveal  that  the  strike  aircraft  (F-18  Hornets)  from  the  carrier  task  force  are 
otherwise  committed  at  the  required  time  so — while  an  EA-6B  Prowler  for  Suppression  of 
Enemy  Air  defense  and  F-18  Hornets  for  fighter  cover  will  be  scheduled  from  the  carrier  task 
force — the  strike  aircraft  (F-15E  Strike  Eagle)  will  be  scheduled  from  the  USAF  forward  air  base 
in  Baranistan.  A  planner  might  consider  submitting  a  request  to  re-task  the  carrier  strike  assets 
to  this  mission,  but  subject  matter  experts  consulted  for  other  projects  have  stated  that  they  are 
generally  reluctant  to  make  such  requests. 

Range  and  endurance  envelopes  are  examined  for  all  selected  aircraft  at  the  Physical  Function 
level  (Figure  10)  and  then  overlaid  on  the  area  of  interest  map  to  assess  whether  the  selected 
aircraft  can  reach  their  respective  operational  areas  and  return  safely.  In  this  scenario,  all  strike 
and  fighter  aircraft  must  refuel  at  least  once.  Refueling  functions  are  found  at  the  Physical 
Function  level  and  followed  to  the  Physical  Resource  level  to  identify  tankers  that  might  satisfy 
this  requirement.  Hyperlinks  from  the  tanker  resources  show  what  refueling  resources  are 
available  and  the  locations  of  refueling  stations.  There  are  two  KC-10  refueling  stations  that 
could  be  used  in  this  scenario;  one  in  north-central  Kartania  and  one  off  the  south  coast  of 
Kartania. 

Figure  22  illustrates  the  use  of  range  overlays  for  the  F-15s  flying  out  of  the  USAF  Forward  base 
in  Baranistan.  The  overlay  centered  on  the  USAF  Forward  base  reveals  that  the  F-15s  cannot 
directly  reach  either  the  target  or  the  tanker  station  off  the  south  coast  of  Kartania,  but  must 
refuel  at  the  tanker  station  in  central  Kartania.  The  overlay  centered  on  the  tanker  station  in 
central  Kartania  reveals  that  they  cannot  return  to  that  station  from  the  target,  but  must  refuel 
again  at  the  tanker  station  off  the  south  coast  of  Kartania.  They  can  then  reach  the  target  and 
refuel  at  the  tanker  station  in  central  Kartania  on  the  way  back  to  the  USAF  Forward  base. 

Similar  use  of  a  range  overlay  would  reveal  that  the  fighter  aircraft  (F-18  Hornet)  from  the 
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Carrier  Task  Force  must  refuel  once  on  the  way  to  the  assigned  loiter  area,  and  once  on  the  way 
back. 

The  Global  Hawk  that  routinely  traverses  the  northern  boundary  of  the  operational  area  is  seen  to 
be  one  that  can  be  tasked  for  this  job.  The  unmanned  aerial  vehicle  is  to  be  diverted  from  its 
regular  route  to  maintain  surveillance  over  the  meeting  site.  Its  surveillance  coverage  capability 
will  be  checked  at  the  Physical  Function  level  and  then  compared  to  the  spatial  layout  of  the 
target  at  the  Mission  Function  level  to  determine  a  standoff  range  for  surveillance  and  to  ensure 
the  surveillance  flight  pattern  is  safe  for  the  vehicle  and  secure  from  observation  by  those  who 
might  warn  the  insurgents  (Figure  21). 

An  examination  of  tail  numbers  at  the  Physical  Function  level  of  the  required  Search  and  Rescue 
aircraft  (the  HH-60G  Pave  Hawk)  reveals  that  one  is  available  from  the  air  asset  inventory  of  the 
carrier  task  force.  It  will  take  up  a  station  for  the  duration  of  the  operation  approximately  1 50 
km  west  of  the  target  in  a  sparsely  populated  area  of  Kartania  (Figure  21)  to  be  readily  available 
for  a  rescue  mission.  Its  mission  station  will  be  close  to  an  in-flight  refueling  station  and  it  will 
be  required  to  refuel  before  it  takes  up  station  and  to  then  set  down  in  a  safe  area  to  await  the 
outcome  of  the  mission.  The  selected  mission  station  is  one  that  permits  ready  access  to  the  area 
in  which  pilots  would  be  at  risk  if  they  were  to  eject  from  their  aircraft,  as  shown  by  the  potential 
Search  and  Rescue  envelope  in  Figure  21. 

Temporal  Coordination 

Once  all  functions  and  specific  assets  have  been  identified,  the  timing  of  the  mission  is  addressed 
at  the  General  Mission  Function  level  by  backtracking  from  scheduled  time  over  target.  Those 
timelines  take  account  of  traversal  time,  in-flight  refueling  time,  and  time  of  arrival  on  station. 

All  assets  have  a  requirement  to  be  on  station  in  advance  of  the  scheduled  attack  time,  and  the 
requirements  differ  for  the  various  assets.  Refueling  slots  must  be  reserved  at  the  necessary 
times,  and  aircraft  must  be  scheduled  to  commence  their  operation  in  time  to  reach  their  assigned 
locations  at  critical  times.  Figure  23  (also  see  Figure  24)  shows  timelines  for  all  assets  at  critical 
locations  such  as  air  refueling  stations.  The  crucial  requirement  is  to  have  all  assets  at  their 
assigned  stations  for  the  strike.  Temporal  coordination  requires  some  care  and  the  plan  must 
ensure  that  temporal  demands  are  not  placed  on  assets  that  will  compromise  their  fuel  resources. 

The  timelines  are  linked  to  individual  assets  to  ensure  that  the  scheduling  of  that  asset  at  various 
locations  is  consistent  with  its  availability  and  the  mission  demand.  Timelines  are  also  inter¬ 
linked  globally  at  strike  time  to  ensure  that  the  plan  has  all  assets  properly  coordinated  as 
follows: 

•  Strike  aircraft  are  to  be  in  a  loiter  area  1 5  minutes  from  the  target  at  0945,  off  the  coast  or 
within  an  area  identified  as  having  no  insurgent  sympathizers  who  might  trigger  a 
suspicion  of  allied  intent. 

•  Timing  of  air-refueling  requirements  with  an  estimated  latest-time-of-attack  at  1 045 
hours  to  ensure  all  aircraft  have  sufficient  fuel  to  remain  on  station  for  the  required  time. 

•  Search  and  rescue  is  to  be  on  station  from  0930  hours  to  1130  hours  with  contingency 
plans  ready  to  rescue  downed  aircrew  in  the  target  approach  area,  the  target  area,  and  the 
target  exit  area.  In  the  case  of  downed  aircrew,  the  unmanned  aerial  vehicle  will  be  re¬ 
tasked  to  aid  the  search  and  rescue  effort. 
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•  Air  surveillance  is  to  commence  at  0900  and  continue  through  to  1200. 

•  The  unmanned  air  vehicle  will  monitor  the  arrival  of  the  insurgents. 

•  The  strike  will  nominally  be  scheduled  for  30  minutes  after  scheduled  start  time  for  the 
meeting  but  this  element  of  the  plan  will  be  subject  to  real-time  adjustment.  The  strike 
will  be  called  in  earlier  if  surveillance  can  confirm  that  the  important  members  of  the 
insurgency  leadership  team  have  arrived,  or  if  there  are  indications  that  the  meeting  is 
breaking  up  early.  In  the  event  that  surveillance  cannot  identify  the  leaders  as  they  arrive 
and  cannot  determine  whether  all  have  arrived,  the  strike  will  proceed  at  30  minutes  after 
the  scheduled  start  time  for  the  meeting.  In  the  event  of  inclement  weather  that  prevents 
visual  monitoring,  infrared  surveillance  will  be  available  as  a  backup. 

•  The  missile  battery  must  be  suppressed  just  prior  to  the  attack,  so  Electronic  Warfare 
aircraft  will  need  to  be  situated  to  move  readily  into  position  just  before  the  strike  aircraft 
are  to  reach  the  target  area.  The  kill  envelope  of  the  missile  will  be  examined  to  ensure 
that  radar  suppression  commences  just  before  the  strike  aircraft  enter  it. 

Air  refueling  is  the  most  complicated  of  the  temporal  coordination  requirements.  In  this 
scenario,  there  are  two  refueling  locations  but  only  one  is  convenient  for  the  Carrier  Task  Force 
assets  and  it  also  must  be  used  to  refuel  the  strike  assets  from  the  USAF  Forward  Base  in 
Baranistan  prior  to  the  attack.  In  addition,  aircraft  on  other  missions  will  also  use  those  refueling 
assets,  so  that  some  of  the  refueling  slots  will  most  likely  be  already  committed  at  the  time  of 
mission  planning.  Since  this  is  a  high-priority  mission  it  would  be  possible  to  bump  other 
refueling  assignments  to  make  way  for  this  mission’s  refueling  requirements,  but  subject  matter 
experts  indicate  that  they  typically  avoid  bumping  other  aircraft  already  assigned  to  refueling 
slots  unless  that  cannot  be  avoided. 


Plan  Review 

Once  the  plan  is  complete,  it  is  reviewed  for  overall  consistency  and  viability.  At  this  point,  the 
human-systems  designer  or  analyst  will  take  on  the  role  of  the  Time  Sensitive  Targeting  cell 
chief  who  will  check  the  plan  and  will  release  it  if  satisfied.  The  cell  chief  will  pay  particular 
attention  to  Operational  Principles  and  Values  to  ensure  that  the  plan  conforms  to  doctrine  and 
guidance.  The  cell  chief  will  apply  qualitative  judgments  to  that  task.  Figure  25  illustrates  a 
scenario  in  which  the  cell  chief  is  generally  satisfied,  but  remains  concerned  about  the  possibility 
of  unacceptable  collateral  damage.  There  may  be  consultation  between  the  cell  chief  and  the 
representative  of  the  Judge  Advocate  General.  They  may  follow  a  hyperlink  from  the  relevant 
spoke  to  a  summary  of  issues  to  investigate  whether  a  strike  proximal  to  a  number  of  sensitive 
structures  of  these  types  is  warranted  for  the  nature  of  this  target. 

The  polar  star  format  shown  in  Figure  25  is  often  used  in  systems  in  which  value  parameters, 
measured  by  technological  means,  are  used  to  adjust  the  lengths  of  the  radials  automatically.  At 
this  stage  in  the  development  of  this  reasoning  space,  it  is  unclear  that  automatic  adjustment  is 
either  possible  or  desirable.  Rather,  the  human-systems  designer  or  analyst  might  introduce  an 
asymmetry  manually  when  one  member  of  a  collaborative  team  notices  an  issue.  That 
asymmetry  will  serve  as  a  reminder  or  as  an  alert,  and  will  be  corrected  when  members  of  the 
collaborative  team  have  dealt  with  the  issue.  In  actual  planning  for  Time  Sensitive  Targeting,  it 
might  be  set  as  asymmetric  by  a  cell  chief  who,  being  dissatisfied  with  the  elements  of  a  plan 
bearing  on  that  value,  wants  to  alert  the  targeteer  to  the  need  to  revise  certain  of  its  elements. 
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Figure  22:  The  use  of  range  rings  to  establish  refueling  requirements 
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Figure  23:  Timelines  for  various  assets  involved  in  the  planned  operation  (see  Figure  24  for  legible  versions  of  key  elements) 
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Figure  24:  Key  elements  of  Figure  23  magnified  to  readable  size  (in  the  approximate  spatial  relationship  of  Figure  23) 
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Prosecution  of  high  priority  targets,  those  that  require  immediate 
response  because  they  pose  a  clear  and  present  danger  to  friendly 
forces  or  are  lucrative,  fleeting  targets  of  opportunity 
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Figure  25:  Purposes  and  Operational  Principles  &  Values  with  a  depiction  that  signifies  a 

concern  for  issues  surrounding  collateral  risk 

Hand  Off 

Once  the  plan  is  verified,  it  will  be  handed  off  for  execution.  It  may  be  posted  to  an  electronic 
information  system  (currently  the  Automated  Deep  Operations  Coordination  System  or  ADOCS) 
and  aircrew  may  be  advised  directly  or  via  the  Airborne  Warning  and  Control  System 
(AWACS).  Ideally,  all  information — target  coordinates  in  particular — will  be  transmitted 
electronically,  but  that  is  not  necessarily  possible  in  all  cases.  In  the  worst  case,  a  targeteer  from 
the  Time  Sensitive  Targeting  cell  may  have  to  relay  coordinates  to  the  AWACS  by  voice  link 
and  those  coordinates  may  then  have  to  again  be  relayed  by  voice  link  from  the  AWACS  to  the 
strike  aircraft.  In  addition,  if  visual  sighting  of  the  target  is  required,  the  pilot  may  have  to  be 
talked  onto  the  target  by  the  AWACS  controller  relaying  a  series  of  identifiable  landmarks  to 
him  or  her  in  real  time. 
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INFORMAL  TEST 


During  the  evolution  of  this  Reasoning  Space  it  seemed  there  would  be  some  value  in  developing 
a  minimal  test  case  as  shown  in  Figure  26.  The  left  panel  depicts  a  fragment  of  the  reasoning 
space  for  air  defense  of  the  naval  task  force.  The  purpose  of  this  subsystem  is  to  coordinate 
defenses  against  a  potential  attack  from  the  air.  The  right  panel  shows  a  potential  trajectory 
through  the  reasoning  space.5  The  potential  trajectory  first  establishes  the  system  purpose,  which 
is  to  coordinate  the  air-to-air  and  surface-to-air  defenses  of  the  task  force.  There  is  a  potential 
threat  from  the  adversary’s  fighter  assets  and  so  the  trajectory  identifies  those  assets  and  then 
examines  their  threat  potential.  Given  an  understanding  of  that  threat  potential,  the  trajectory 
then  identifies  the  defensive  resources  of  the  naval  task  force,  first  aircraft  and  their  capabilities 
and  then  missiles  and  their  capabilities.  All  capabilities  are  compared  in  terms  of  defensive 
coverage  demand  versus  defensive  capability  and  finally  assessed  in  relation  to  the  risk  balance 
for  task  force  protection  that  results  from  this  configuration  of  assets. 
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Figure  26:  A  fragment  of  the  reasoning  space  for  air  defense  of  a  naval  task  force  (left 
panel)  and  a  potential  trajectory  through  that  reasoning  space  (right  panel) 


5  Note  that  this  trajectory  has  been  created  as  an  illustration  and  it  remains  important  to  collect  actual  trajectories 
with  individuals  knowledgeable  in  the  domain  who  have  not  been  involved  in  the  developments  reported  here.  An 
actual  evaluation  that  resulted  in  sensible  trajectories  would  provide  considerable  support  for  the  ideas  outlined  in 
this  report.  » 
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RESOURCE  DESCRIPTIONS  VIA  EMBEDDED  HYPERLINKS 

Earlier  discussion  referred  to  hyperlinks  embedded  in  depictions  of  resources  that  would  lead  to 
a  succinct  and  focused  description  of  the  resource.  The  documents  I  consulted  in  the 
development  of  this  reasoning-space  concept  had  a  considerable  amount  of  detail  about  specific 
resources  that  might  be  of  general  interest,  but  that  impeded  the  review  of  information  critical  to 
mission  planning  because  it  was  irrelevant  to  the  targeting  task.  The  embedded  hyperlinks  for  a 
reasoning  space  should  access  a  succinct  but  informative  description  of  details  relevant  to  the 
planning  tasks  to  be  supported  by  that  space.  Information  not  specifically  relevant  to  the  work 
supported  by  the  reasoning  space  should  be  excluded,  although  some  care  must  be  taken  in 
deciding  what  is  relevant  and  what  is  not.  Information  should  not  be  excluded  merely  because  it 
is  not  relevant  to  a  specific  scenario.  Different  planning  tasks  will  inevitably  require  access  to 
different  constellations  of  information,  and  the  reasoning  space  should  include  all  information 
that  can  be  relevant  to  the  full  task  range  (i.e.,  the  work).  In  addition,  those  documents  should  be 
consistently  structured  and  formatted  to  support  side-by-side  comparisons  between  resources  that 
permit  different  functional  capabilities  to  be  readily  assessed. 

Early  in  the  development  of  the  reasoning  space,  I  conceptualized  that  description  as  a  text 
document.  Pop-up  summaries  of  that  type  are  shown  for  the  F- 15  Strike  Eagle  (Figure  27),  the 
A-IO/OA-IO  Thunderbolt  (Figure  28),  the  Global  Hawk  UAV  (Figure  29),  and  the  KC-135  and 
the  KC-10  tanker  aircraft  (Figure  30).  Note  that  the  information  content  of  these  examples, 
abstracted  from  an  unclassified  web  site  (http://www.af.mil/factsheets/).  remains  incomplete. 

For  example,  three  entries  in  the  callout  for  KC-10  (Refueling  time  for  each  aircraft,  Number  of 
aircraft  serviced  concurrently,  Maximum  time  on  station)  and  two  for  the  KC-135  are 
placeholders  for  information  that  is  probably  important  but  is  not  available  from  the  documents 
consulted  for  this  project. 

Somewhat  late  in  this  project,  it  became  evident  that  these  pop-up  summaries  could  be  structured 
in  the  form  of  an  Abstraction-Decomposition  space.  At  this  time,  my  thought  is  that  the  more 
detailed  descriptions  of  specific  resources  should  cover  the  bottom  three  levels  of  abstraction 
(Physical  Resources  &  Constraints,  Technical  Functions  &  Contextual  Effects,  and  General 
Mission  Functions)  and  where  possible,  should  contain  graphics  rather  than  text.  Many  of  the 
graphics  shown  previously  would  be  suitable  (e.g.,  Figure  10  and  Figure  17).  This  vision  of  how 
to  represent  a  summary  of  the  specific  resources  came  to  me  too  late  in  the  project  to  develop  it 
fully,  but  it  offers  one  opportunity  for  further  development. 
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hyperlinked  summary  (shown  here  as  a  pop  up)  of  the  F-15  Strike  Eagle 
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Figure  28:  A  hyperlinked  summary  (shown  here  as  a  pop  up)  of  the  A-10/OA-10  Thunderbolt 
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Figure  29:  A  hyperlinked  summary  (shown  here  as  a  pop  up)  of  the  Global  Hawk  UAV 
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Figure  30:  A  hyperlinked  summary  (shown  here  as  a  pop  up)  of  the  KC-135  and  the  KC-10  tanker  aircraft 


SYSTEM  SPECIFICATIONS 


There  is  a  requirement  for  this  project  to  outline  the  specifications  for  the  reasoning  space  that 
has  been  developed.  It  is  common  in  technical  design  projects  to  write  specifications  in  the  form 
of  a  text  document.  For  design  of  cognitive  systems  at  least,  that  is  an  ungainly — and  inevitably 
unsatisfactory — strategy.  One  of  the  forces  that  motivated  this  project  was  to  find  a  better  way 
of  transmitting  specifications.  Consistent  with  Lintem  (2005),  this  work  assumes  that 
engagement  on  representational  forms  will  be  much  more  effective  than  a  text  document  in 
helping  software  engineers  design  and  write  code  to  satisfy  the  requirements. 

It  would  negate  the  foundational  philosophy  behind  this  work  to  provide  specifications  for  the 
reasoning  space  in  the  format  most  often  used  for  specification  of  a  technological  product. 
Instead,  the  preferred  strategy  is  to  specify  requirements  via  a  storyboard  narrative  that 
demonstrates  how  the  human  agents  will  interact  with  each  other  and  with  the  technological 
features  of  the  system  (Lintem,  2005).  To  that  end,  the  use  narratives  provided  above  should  be 
viewed  not  only  as  an  illustration  of  how  the  reasoning  space  will  be  used,  but  also  as  a 
specification  for  how  it  should  be  implemented  in  software — albeit  a  partial  specification  as  yet. 
It  is  not  the  proper  role  of  cognitive  engineer  to  advise  a  software  engineer  how  to  structure  or 
write  code,  but  rather  to  ensure  that  the  software  engineer  understands  the  functional 
requirements.  That  is  far  better  accomplished  by  an  evocative  storyboard  than  by  a  set  of  written 
instructions. 

One  dimension  of  specification  not  treated  in  the  storyboard  narrative  relates  to  how  the 
reasoning  space  might  be  implemented  as  a  three-dimensional  workspace.  Although  discussion 
of  three  dimensionality  can  evoke  thoughts  of  holographic  or  binocular  displays,  it  is  thought  at 
this  time  that  perspective  views  represented  on  a  flat  panel  screen  will  satisfy  the  requirement — 
although  it  is  likely  that  a  panel  much  larger  than  the  typical  size  will  be  desirable.  An  electronic 
tabletop  of  the  type  depicted  in  Figure  3 1  may  be  required  to  allow  a  comprehensive  overview  of 
the  total  space  of  an  extensive  knowledge  domain.  Because  of  limited  resources,  the  three- 
dimensional  format  has  not  been  illustrated  in  this  project.  The  reasoning  space  has  instead  been 
depicted  as  a  series  of  two-dimensional  panels — see  Figure  5,  (also  Figure  25),  Figure  8,  Figure 
9,  and  the  set  that  depicts  different  views  of  General  Mission  Functions,  Figure  21,  Figure  22  and 
Figure  23. 
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Figure  31:  A  depiction  of  an  electronic  table  top  that  might  be  used  to  explore  a  full 
reasoning  space  for  a  complex  socio-technical  system 


IMPLICATIONS  OF  THE  REASONING  SPACE  FOR  SYSTEM  DESIGN 

What  is  striking  is  the  disconnect  between  operations  as  viewed  at  the  top  and 
operations  as  implemented  on  the  front  line. 

Weick  and  Sutcliffe  (2001),  p  14 

The  reasoning  space  is  not  a  design  artifact  in  itself  but  a  domain  knowledge  system.  Its 
development  is  based  on  the  assumption  that  those  who  know  the  functional  structure  of  the 
system  and  the  processes  that  operate  within  it  will  be  better  placed  to  design  support  systems  for 
it  or  to  redesign  it.  As  Vicente  (1999)  argues,  the  complexity  of  a  technological  support  must 
reflect  the  complexity  of  the  work.  This  view  is  consistent  with  the  Law  of  Requisite  Variety 
(only  variety  can  destroy  variety,  Ashby,  1957,  p207),  here  taken  to  mean  that  only  variety  can 
control  variety.  In  other  words,  the  controller  must  have  as  much  variety  as  the  system  it 
controls  rhttp://artsandscience.concordia.ca/edtech/ETEC606/requisite.htmll:  the  functional 
scope  and  granularity  of  a  workspace  must  match  the  operational  complexity  of  the  work.  While 
Ashby  supported  his  law  with  a  rigorous  mathematical  analysis,  it  also  makes  intuitive  sense;  a 
system  that  has  more  behavioral  options  than  its  controller  always  has  the  potential  to  surprise  or 
outmaneuver  its  controller. 

The  central  problem  to  be  addressed  by  use  of  the  reasoning  space  is  resolution  of  the  disparity 
between  design  conceptualization  and  the  natural  strategies  available  to  operational  personnel. 
Typically,  those  who  design  major  systems  fail  to  appreciate  the  subtle  complexities  of  the  work. 
Within  the  socio-technical  design  world,  there  is  what  Weick  and  Sutcliffe  (2001)  refer  to  as 
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“mindlessness.”  Their  discussion  is  specifically  about  the  conceptual  disengagement  of 
management  from  operations.  They  argue  that  a  socio-technical  system  needs  a  common 
language  that  is  understood  at  all  levels;  a  language  that  can  be  used  to  understand  operational 
complexity.  In  management,  the  problem  surfaces  through  disparity  between  management  and 
operational  staff  in  understanding  what  it  means  to  be  sensitive  to  operations.  For  management 
operational  issues  relate  to  organizational  structures  and  financial  accountability,  while  for 
operational  staff  they  relate  to  issues  surrounding  execution  of  the  work. 

In  design  of  socio-technical  systems,  there  is  a  similar  disparity.  Designers  are  concerned  with 
developing  a  technical  system  that  works,  while  operational  personnel  are  concerned  with  how 
these  tools  support  their  work  activities.  In  principle,  there  is  no  need  for  these  two  perspectives 
to  be  at  odds,  but  for  there  to  be  an  effective  synergy  designers  need  to  understand  how  their 
system  integrates  with  the  complexity  of  the  work.  Designers  typically  have  an  impoverished 
view  of  the  nature  of  the  work,  and  the  technological  work  supports  they  develop  reflect  that 
view.  The  preceding  discussion  has  outlined  how  a  designer  might  proceed  to  become  familiar 
with  the  operational  complexities  of  the  work  (its  functional  scope  and  granularity)  as  a  means  of 
developing  a  more  realistic  (Weick  and  Sutcliffe  use  the  term,  “more  mindful”)  appreciation  of 
operational  activities.  To  supplement  that  discussion,  this  section  of  the  report  illustrates  how 
that  operational  knowledge  can  guide  system  redesign. 

Three  examples  are  outlined  below  to  illustrate  how  use  of  the  reasoning  space  might  help  a 
design  team  envision  an  innovative  work  support  and  then  lead  that  design  effort  in  a  productive 
direction.  The  examples  chosen  for  that  illustration  emphasize  applications  from  the  domain  of 
cognitive  engineering,  but  the  reasoning  space  is  intended  to  be  more  general  than  that  and 
should  be  useful  for  designers  from  other  technical  disciplines.  Cognitive  engineering  examples 
have  been  selected  for  illustration  primarily  because  I,  as  a  cognitive  engineer,  could  not  develop 
credible  illustrations  that  emphasize  the  contributions  of  other  disciplines.  It  would  nevertheless 
be  valuable  in  further  development  of  this  reasoning  space  to  engage  members  of  other  technical 
disciplines  to  assess  whether  their  work  would  also  benefit. 

In  the  review  of  these  illustrations,  it  should  be  noted  that  the  reasoning  space  is  about  specifics 
of  a  work  domain  and  that  no  design  effort  can  proceed  only  on  that  basis.  A  designer  brings 
experience  and  knowledge  about  general  principles,  and  any  successful  design  effort  will 
combine  that  sort  of  knowledge  with  specific  knowledge  about  the  work  domain.  In  addition,  it 
should  be  recognized  that  there  are  certain  creative  elements  in  the  process  of  design  that  defy 
logical  or  explicit  description.  The  best  that  can  be  said  is  that  creative  innovation  emerges 
through  an  abductive  leap.  Creative  design  is  not,  as  some  seem  to  imply,  a  combinatorial  search 
through  a  high-dimensional  state  space.  Instead,  innovation  builds  on  extensive  knowledge  of 
and  familiarity  with  the  design  space  through  a  process  in  which  designers  become  aware  of  new 
possibilities  via  what  can  be  best  described  as  insight. 
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Team  Design 


Our  primary  recommendation  was  to  reduce  the  number  of  people  in  the  technical 
support  center.  The  excessive  staffing  produced  confusion  about  roles  and 
functions  and  led  to  the  "social  loafing  ”  phenomenon  in  which  team  members 
come  to  define  their  roles  more  narrowly  and  to  assume  that  others  will  handle 
new  tasks. 

Klinger  &  Klein,  1999,  p  23 

Lean  manning  has  become  an  important  design  goal  for  many  military  systems.  The  problem  is 
generally  seen  as  one  of  the  combining  work  roles  without  inducing  cognitive  overload,  and 
automation  is  promoted  as  the  primary  technological  solution.  This  strategy  does  not,  however, 
offer  anything  useful  for  the  concept  refinement  phase  of  systems  acquisition.  A  strategy  based 
on  the  global  functional  requirements  would  be  a  more  effective  approach. 

The  recommended  strategy,  motivated  in  part  by  work  that  has  shown  increased  efficiency  of 
teamwork  with  well  designed  reductions  in  manning  (Klinger  and  Klein,  1 999),  is  to  first 
establish  an  efficient  global  functional  structure  and  to  then  descend  through  organizational 
layers,  defining  functional  structures  in  terms  of  functionally  oriented  organizational  units,  until 
work  is  allocated  to  specific  agents,  either  human  or  technological.  To  ensure  that  the 
illustration  outlined  here  is  concrete,  the  existing  functional  structure  of  the  Air  Operations 
Center,  as  shown  in  Figure  32,  is  accepted.  Time  Sensitive  Targeting,  as  discussed  earlier  in  this 
report,  is  identified  more  generically  as  a  real-time  planning  and  targeting  function.  It  is  located 
in  the  Offensive  Operations  unit  within  the  Combat  Operations  Division. 

The  design  goal  is  to  establish  viable  work  packages,  where  a  work  package  is  defined  as  that 
constellation  of  work  products  that  can  be  delivered  by  one  human  agent  with  the  support  of  any 
assistive  technological  functions  that  can  be  provided.  In  any  complex  socio-technical  system, 
work  packages  will  be  interdependent  but  they  should  be  designed  as  far  as  possible  to  be 
modular  in  the  sense  that  their  reliance  on  other  system  products  should  be  minimized. 
Communication  overhead  offers  possibly  the  best  illustration  of  this  principle.  It  is  well  known 
that  communication  between  agents  places  a  load  on  the  system.  Where  at  least  one  of  those 
agents  is  human,  the  requirement  for  communication  increases  coordinative  challenges  and 
cognitive  load.  Efficiencies  will  be  realized  when  the  complexity  and  frequency  of 
communications  undertaken  by  human  agents  are  minimized. 

By  developing  a  plan  within  the  reasoning  space,  a  human-systems  designer  or  analyst  will  gain 
an  idea  of  what  type  of  work  is  involved,  and  how  time  consuming  and  cognitively  intensive  it 
is.  For  the  narrative  in  the  section  titled  A  Planning  Trajectory  for  a  Human-Systems 
Designer,  a  single  person  does  the  planning  and  then  passes  the  plan  to  the  cell  chief  for  review. 
In  practice,  there  are  at  least  three  members  of  a  Time  Sensitive  Targeting  cell  to  do  this  work,  a 
targeteer,  a  rerole  officer,  and  an  attack  coordinator  (Lintem,  2005).  The  human-systems 
designer  or  analyst  could  use  his  or  her  planning  experience  gained  in  working  through  the 
reasoning  space  to  consider  how  a  plan  in  development  is  handed  from  one  member  to  another, 
and  could  consider  whether  a  different  configuration  of  the  work  is  desirable. 
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Figure  32:  Functional  organization  of  the  Air  Operations  center 

The  illustration  of  Figure  33  suggests  an  organizational  concept  for  real-time  planning  and 
targeting.  A  human-systems  designer  or  analyst  who  has  studied  the  reasoning  space  and 
worked  through  narratives  would  now  understand  what  tasks  have  to  be  accomplished  within  the 
process  of  real-time  planning  and  targeting.  By  working  through  the  scenario  narratives  in 
particular,  he  or  she  would  have  come  to  understand  the  linkages  between  the  various  subtasks. 

It  should  be  possible  at  this  stage  to  propose  how  the  total  work  package  might  be  modularized.6 
Two  or  three  different  structures  might  seem  possible. 

There  is  often  a  time  constraint  on  developing  a  plan.  The  human-systems  designer  or  analyst 
could  use  the  reasoning  space  to  consider  whether  current  and  alternative  configurations  can 
result  in  the  plan  being  finished  within  that  constraint.  In  a  previous  project,  one  subject  matter 
expert  spoke  of  an  arrangement  in  which  the  duties  of  the  targeteer  and  the  rerole  officer  were 
combined  in  a  manner  that  permitted  more  efficient  planning,  and  also  permitted  planning  for 
two  targets  simultaneously.  The  designer  might  use  the  reasoning  space  to  work  through  the 
planning  process  under  each  of  these  configurations,  possibly  with  the  collaborative  assistance  of 
a  subject  matter  expert  much  as  has  been  done  by  Naikar,  Pearce,  Drumm  and  Sanderson  (2003) 


6  Note  that  the  possibilities  for  redesign  emerge  via  abductive  reasoning,  otherwise  known  as  insight.  The  role  of 
the  reasoning  space  in  this  process  is  to  help  the  designer  become  sufficiently  knowledgeable  that  the  insights  reflect 
the  complexity  of  the  work. 
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with  tabletop  analysis  in  their  work  on  team  design  for  the  Australian  airborne  early  warning 
system.  This  process  will  help  the  human-systems  designer  or  analyst  think  about  different  ways 
of  configuring  the  planning  team  and  of  providing  suitable  collaborative  support. 


REAL-TIME  PLANNING  &  TARGETING 


Workstation  visualization 

Many  of  the  depictions  in  the  reasoning  space  described  here  could  potentially  provide  effective 
visualizations  within  an  actual  Time  Sensitive  Targeting  cell.  For  example,  an  integrated 
depiction  of  the  target  area,  as  shown  in  Figure  4,  may  enhance  the  sensitivity  of  the  planner  to 
the  collateral  damage  constraints,  the  problems  associated  with  enemy  ground  offenses,  and 
features  that  may  be  used  to  help  the  attack  pilot  locate  the  target.  A  formal  workspace  as 
developed  by  Lintem  (2006)  for  counterinsurgency  planning  could  possibly  be  developed,  and 
much  of  what  has  been  shown  in  Figure  5  through  Figure  25  could  be  useful  for  that  workspace. 
These  ideas  would  constitute  design  hypotheses  about  what  might  be  useful,  and  the  designer 
(after  first  laying  them  out  in  some  configuration)  could  explore  their  value  by  working  through 
scenario  narratives  within  the  reasoning  space  with  the  assistance  of  one  or  more  subject  matter 
experts  who  would  most  likely  suggest  variations  and  extensions. 
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Large-Screen  Wall  Displays  for  an  Air  Operations  Planning  Center 

There  has  been  relatively  little  thought  about  what  sort  of  content  the  large-screen  wall  displays 
in  an  Air  Operations  planning  center  should  contain.  Following  Clark’s  views  on  ubiquitous 
computing  (Clark,  2003),  I  suggest  that  common  wall  displays  should  provide  contextual  and 
structural  information  (for  maintenance  of  global  situation  awareness)  that  operational  staff 
would  assimilate  with  little  conscious  effort.  That  information  should  not,  however,  capture 
attention.  In  a  previous  project  (Lintem,  2005)  a  subject  matter  expert  stated  that  the  large 
screen  display  of  an  Air  Operations  planning  center  in  which  he  worked  carried  a  continuous 
feed  from  a  commercial  24-hour  news  channel.  Many  on  the  operational  floor  would  watch  that 
and,  in  particular,  video  of  weapons  strikes  captured  the  attention  of  many.  In  a  similar  vein, 
direct  feeds  from  unmanned  air  vehicles  or  from  strike  aircraft  might  be  popular. 

Popularity  is  not  the  essential  aim.  While  observations  of  subject  matter  experts  are  valuable, 
they  should  not  be  permitted  to  enslave  design.7  Ideally,  large-screen  wall  displays  will  offer 
information  that  provides  context  for  the  more  focused  information  that  can  be  accessed  through 
an  individual  workstation  for  a  specific  planning  task.  Hypotheses  about  what  constitutes 
contextual  information  and  would  be  of  general  use  as  ubiquitous  or  orienting  information  can  be 
identified  by  examination  of  resources  and  constraints  as  shown  in  Figure  8. 

Any  information  selected  for  wall  screen  presentation  should  be  presented  pictorially  for  tacit 
assimilation.  A  display  of  current  or  anticipated  weather  patterns  throughout  the  area  of 
operations,  a  display  of  the  area  of  operations  with  key  features  highlighted,  or  a  display  of 
ongoing  air  and  surface  missions  (as  in  the  old-style  sand  table)  may  be  useful.  The  general 
patterns  would  provide  context  for  any  specific  planning  task  and  would  support  situation 
awareness.  Where  more  specific  details  are  required,  a  particular  staff  member  could  access 
those  through  his  or  her  own  workstation. 


SUMMARY 

The  development  of  the  reasoning  space  and  the  discussion  above  have  been  motivated  by  the 
view  that  the  most  serious  impediment  to  the  effective  design  of  a  complex,  socio-technical 
system  like  an  Air  Operations  Center  is  the  difficulty  of  developing  innovative  ideas  that  address 
important  work  constraints  as  that  work  unfolds  in  actual  operations — rather  than  as  it  is 
envisaged  from  the  relative  safety,  comfort,  and  organization  of  a  high  technology  design 
environment.  The  primary  purpose  of  the  reasoning  space  is  to  bring  into  that  high  technology 
design  environment  an  appreciation  of  the  complex  demands  and  the  operational  constraints  that 
interact  to  generate  an  often-confusing  work  environment  that  cannot  be  understood  in  depth  or 
at  any  meaningful  level  in  the  abstract. 

Lintem  (2005)  has  promoted  the  development  of  a  rapid  prototyping  tool  that  would  enable  a 
multidisciplinary  design  team  to  explore  the  implications  of  various  cognitive  specifications  and 
evaluate  various  system  configurations.  Design  is  viewed  as  a  creative  dialog  among  members 


7  One  can  imagine  that  commercial  films  would  be  popular  but  they  would  not  support  the  work  at  hand  and  would 
serve  primarily  as  a  distraction. 
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of  a  design  team  with  different  areas  of  expertise.  The  reasoning  space  discussed  in  this  report 
offers  a  means  for  an  individual  designer  to  develop  a  personal  dialog,  and  to  then  share  it  and 
extend  it  in  interaction  with  others  as  they  also  work  collaboratively  through  the  reasoning  space. 

The  purpose  of  the  reasoning  space  is  to  help  a  design  team  identify  areas  of  work  where  some 
form  of  work  support  may  assist  the  workflow.  The  knowledge  acquired  from  the  space  should 
then  help  the  design  team  conceptualize  a  work  support  that  would  be  effective.  Within  a  socio- 
technical  environment  in  which  operational  personnel  discard  many  new  support  systems 
because  they  are  too  cumbersome  or  too  poorly  conceptualized,  effective  use  of  the  reasoning 
space  in  this  manner  could  constitute  a  huge  advance.  Whether  the  reasoning  space  can,  in  fact, 
be  effective  in  these  terms  can  only  be  confirmed  in  use;  designers  must  use  it  to  develop  work 
supports  and  then  those  work  supports  must  find  favor  with  operational  staff  and  result  in 
improved  workflow  for  them. 

As  noted  earlier  in  this  report,  Cognitive  Systems  Engineering  is  an  analysis  and  design 
discipline  for  Human-System  Integration  within  socio-technical  systems.  Analytic  methods 
focus  on  the  complexities  of  work  by  identifying  why  the  work  is  cognitively  difficult,  the  types 
and  levels  of  expertise  required,  the  functional  or  informational  structure  of  the  work  domain,  the 
tasks  or  processes  employed,  the  means  by  which  workers  develop  situation  awareness,  and  the 
means  by  which  workers  coordinate  and  communicate.  A  deep  understanding  of  the  work 
domain,  as  facilitated  through  use  of  the  reasoning  space,  should  help  a  designer  formulate 
solutions  to  such  issues. 
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BACKGROUND 


This  tutorial  was  developed  by  Dr.  Gavan  Lintem.  Dr.  Lintem  serves  Chief  Scientist  at  General 
Dynamics  -  Advanced  Information  Engineering  Services.  Dr.  Lintem  is  a  Subject  Matter  Expert 
in  the  field  of  Cognitive  Systems  Engineering.  He  previously  served  as  Director  of  Human 
Factors  for  the  Air  Operations  division  of  Australia’s  Defense  Science  and  Technology  systems 
research  where  he  managed  and  built  a  program  in  Cognitive  Systems  Engineering. 

INTRODUCTION 

Work  Domain  Analysis  identifies  the  functional  structure  of  a  socio-technical  system.  That 
functional  structure  will  encompass  properties  ranging  from  object  descriptions,  through  specific 
and  general  functions,  to  values  and  specifications  of  system  purpose.  It  will  encompass 
functional  properties  that  result  from  design  intent  but  in  addition,  functional  properties  that  may 
not  have  been  intended  but  instead  were  discovered  by  operators,  and  both  desirable  and 
undesirable  functional  properties  generated  because  of  interaction  with  context  or  environment. 

Work  Domain  Analysis  identifies  structure  independently  of  activity.  It  can  be  likened  to  a  map 
that  lays  out  the  structure  of  a  geographic  environment.  Activity  is  important,  but  neither  a  Work 
Domain  Analysis  nor  a  map  addresses  that.  However,  both  provide  important  leverage  into 
planning  of  activity  by  laying  out  the  resources  available  for  action  and  the  constraints  on  action. 
(Work  Domain  Analysis  is  part  of  a  larger  analytic  framework,  Cognitive  Work  Analysis,  in 
which  the  later  stages  that  reference  the  product  of  Work  Domain  Analysis  deal  with  activity). 

The  approach  might  be  clarified  by  consideration  of  the  meaning  of  the  terms  function  and 
process.  These  are  troubling  terms  in  engineering  and  science  because  their  range  of  usage  is 
broad  and  they  have  overlapping  meanings.  Within  Cognitive  Work  Analysis,  Vicente  (1999) 
has  given  them  constrained  meanings  that  map  onto  the  needs  of  this  analytic  framework.  A 
function  is  a  structural  property  of  the  work  domain.  A  process  is  a  mechanism  by  which  the 
behavior  of  the  system  is  produced.  This  distinction  is  unusual  and  no  other  strategy  of  cognitive 
analysis  makes  it  explicit.  An  underlying  assumption  of  Cognitive  Work  Analysis  is  that  the 
separation  of  structure  from  activity  helps  bring  an  important  source  of  order  to  the  analysis  of 
complex,  socio-technical  systems. 

The  product  of  Work  Domain  Analysis  is  an  Abstraction-Decomposition  map;  a  two- 
dimensional  matrix  that  distributes  functions  across  levels  of  abstraction  (object  descriptions, 
physical  functions,  purpose  related  functions,  values  and  system  purposes)  and  across  degrees  of 
decomposition.  By  convention,  abstraction  is  represented  on  the  vertical  dimension  and 
decomposition  on  the  horizontal  dimension. 

A  major  contribution  of  Work  Domain  Analysis  is  that  it  identifies  means-end  relationships 
between  functions  at  different  levels  of  abstraction.  A  means-end  relation  reveals  the  functions 


8  Even  within  Systems  Engineering,  where  this  sort  of  distinction  would  seem  to  offer  an  advantage,  Functional 
Analysis  as  discussed  in  many  texts  and  reports  (e.g.,  as  in  Blanchard  and  Fabiycky,  1990)  is  a  functional  flow 
analysis,  essentially  a  process  analysis. 
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at  one  level  that  must  be  used  for  satisfaction  of  a  function  at  a  higher  level.  In  most  cases,  a 
constellation  of  functions  at  the  lower  level  will  be  required  to  satisfy  any  function  at  a  higher 
level. 

In  this  discussion  of  means-end  relations,  the  reference  is  specifically  to  resources  that  will  be 
used  to  satisfy  a  functional  requirement.  It  is  often  said  that  means-end  relations  describes  how  a 
function  is  achieved  but  the  word  how  implies  a  reference  to  both  resources  and  activity.  In 
principle,  a  means-end  relation  could  specify  either  and  it  could  even  be  useful  to  have  means- 
end  relations  specify  both.  However,  the  standard  approach  to  Work  Domain  Analysis 
specifically  excludes  any  form  of  reference  to  activity  and  so  a  means-end  relation  refers  only  to 
the  resources  that  must  be  used  to  achieve  ends  (which  is  actually  consistent  with  the  accepted 
definition  of  a  means  test).  You  should  note  this  carefully  because  it  is  a  source  of  considerable 
confusion  within  discussions  about  Work  Domain  Analysis. 

The  remainder  of  this  brief  tutorial  will  focus  on  the  Abstraction-Decomposition  map; 
descriptions  of  how  it  should  look  and  hints  about  how  to  construct  it.  Note  however,  that  the 
construction  of  an  Abstraction-Decomposition  map  requires  considerable  knowledge  about  the 
system  under  consideration.  The  assembly  of  that  knowledge  constitutes  a  major  Knowledge 
Acquisition  effort,  typically  an  extraction  of  relevant  details  from  document  reviews  and 
discussions  with  subject  matter  experts.  There  is  little  in  this  tutorial  that  tells  you  how  to  do  that 
but  if  you  need  further  advice  on  that  aspect  of  Work  Domain  Analysis,  I  can  suggest  sources. 

THE  ABSTRACTION-DECOMPOSITION  MAP 

The  concept  of  abstraction  is  depicted  in  Figure  A-l .  In  this  example,  comfort  is  the  most 
abstract  function  and  is  enabled  by  heating  (a  physical  function)  which  is  enabled  by  the  furnace 
(a  physical  object).  An  important  aspect  not  depicted  in  Figure  A-l  is  that  both  comfort  and 
heating  could  be  enabled  by  other  sorts  of  resources  not  depicted  here.  The  same  abstraction 
shown  in  Figure  A-l  is  extended  in  Figure  A-2  to  depict  decompositions  that  might  be  useful  for 
the  analysis  of  a  heating  system. 


Figure  A-l.  A  simple  depiction  of  an  abstraction  relationship  with  means-end  links  shown 
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|  Physiological 

/L  Stability 


Figure  A-2.  A  simple  depiction  of  decomposition  relationships,  building  on  the  abstraction 

of  Figure  A-l 

The  knowledge  representation  for  Work  Domain  Analysis,  the  Abstraction-Decomposition  map, 
provides  the  foundation  for  the  design  of  a  radically  new  system  form.  This  map  represents 
functional  properties  of  the  work  domain  (objects,  resources,  constraints,  purposes)  in  a  two- 
dimensional  matrix  (Figure  A-3).  Each  node  represents  a  function.  The  vertical  dimension 
represents  the  dimension  of  abstraction  and  the  horizontal  dimension  shows  varying  levels  of 
decomposition  (system,  unit,  component,  part). 

The  abstraction  dimension  of  an  Abstraction-Decomposition  map  is  typically  organized 
(proceeding  from  top  to  bottom)  through  the  hierarchy  of: 

•  The  System  Purpose;  the  particular  purpose  that  is  the  focus  of  analysis 

•  Values  and  Priorities,  Abstract  Functions,  Balances;  functions  that  encapsulate 
human  and  social  values  (e.g.,  safety-productivity  tradeoffs,  concerns  about  collateral 
damage,  conservation  concerns  with  regard  to  own  personnel  and  resources)  and 
thereby  constrain  the  space  of  acceptable  action 

•  Purpose-Related  Functions;  the  general  functions  that  will  satisfy  the  system 
purpose 

•  Physical  Functions  and  Physical  Effects;  the  effects  or  processes  supported  by  or 
generated  by  physical  systems  or  objects 

•  Physical  Properties;  the  physical  properties  of  objects  and  devices  such  as  location, 
layout,  appearance  and  shape 

Further  discussion  of  the  meaning  of  these  labels  to  the  levels  of  abstraction  and  tips  for 
constructing  an  Abstraction-Decomposition  map  are  presented  in  Appendix  B  of  this  report. 


56 


Figure  A-3.  The  standard  two-dimensional  format  of  an  Abstraction-Decomposition  map 

Abstraction  levels  are  connected  by  means-end  links,  shown  in  Figure  A-3  as  solid,  two-headed 
arrows.  This  form  is  used  to  indicate  the  reciprocity  between  related  functions  at  different  levels; 
a  function  is  enabled  or  supported  by  functions  to  which  it  is  connected  at  lower  levels  (its 
means)  and  conversely,  a  function  is  implemented  to  support  or  enable  functions  to  which  it  is 
connected  at  higher  levels  (its  ends).  Decompositions  within  levels  are  represented  by  single¬ 
headed,  dashed  arrows  with  the  arrow  pointing  in  the  direction  of  the  decomposition. 

You  might  need  to  deviate  from  the  standard  representational  form  for  the  Abstraction- 
Decomposition  map  because  the  multiple-column  format  cannot  easily  be  made  legible  in  a 
normal-size  document  page.  An  alternate  form,  shown  in  Figure  A-4  relies  on  dashed  arrows  to 
indicate  decompositions  within  abstraction  levels  and  conversely,  enclosures  to  indicate 
aggregations.  A  decomposition  of  system  into  units  is  shown  at  the  abstraction  level  of  Purpose- 
Related  Functions  and  a  decomposition  of  a  unit  into  components  and  an  aggregation  of 
components  into  unit  are  shown  at  the  abstraction  level  of  Physical  Functions  and  Physical 
Effects  (at  this  level,  the  term  function  is  reserved  for  the  consequences  of  engineered  artifacts 
while  the  term  effect  is  used  to  refer  to  the  consequences  of  natural  phenomena  such  as  weather). 
Decompositions  should  continue  to  the  level  of  operational  relevance. 
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Figure  A-4.  An  alternate  format  for  an  Abstraction-Decomposition  map 


Figure  A-5  offers  a  tutorial  example  of  an  Abstraction-Decomposition  map  that  shows  some  of 
the  functional  elements  for  home  heating  that  contribute  to  comfort  and  health.  Several  features 
should  be  noted: 

•  The  different  types  of  function  terms  at  each  level. 

•  The  means-end  relations  (two-headed  arrows  between  levels) 

o  The  reason  for  a  function  at  one  level  is  shown  by  its  connection  to  one  or  more 
functions  at  the  next  highest  level. 

o  The  structural  means  of  satisfying  a  functional  requirement  are  shown  by  the 
means-end  links  to  functions  at  the  next  lowest  level 

•  Decompositions  shown  by  dashed,  single-headed  arrows  within  levels. 

•  Interdependencies  shown  by  the  crossings  of  the  means-end  relations.  Links  from  passive 
objects  (insulation,  enclosures)  to  heat  extraction  suggest  interdependency,  which  here  is 
that  poor  passive  systems  will  exacerbate  the  load  on  the  active  system  and  will  possibly 
cause  an  overload  of  the  system. 

•  The  means-end  link  from  the  passive  to  the  active  leg  goes  to  the  Heat  Extraction  process 
and  not  to  the  overload  circuit.  This  is  an  important  feature.  We  want  to  pose  the  issue  at 
a  general  level.  Cooling  can  be  accomplished  by  different  means  (evaporative  coolers  are 
effective  in  low  humidity  environments  would  be  terrible  where  humidity  is  high). 

•  The  heat  extraction  process  might  overload  because  of  poor  insulation.  Do  you  correct 
this  problem  with  a  higher  capacity  cooling  system  or  better  insulation?  Work  Domain 
Analysis  does  not  answer  this  question,  but  it  does  represent  the  options  in  a  manner  that 
helps  you  understand  what  you  might  do. 
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Figure  A-5.  An  Abstraction-Decomposition  map  that  shows  some  of  the  functional 
elements  for  home  heating  that  contribute  to  comfort  and  health 


AN  ABSTRACTION-DECOMPOSITION  MAP  IS  A  DESIGN  ARTIFACT 

An  Abstraction-Decomposition  map  is  not  a  design  or  even  a  design  specification  but  rather  a 
design  artifact.  It  organizes  information  in  a  systematic  manner  that  will  support  design.  For 
example,  it  can  be  used  to  specify  the  information  requirements  of  a  work  domain.  Each  node  in 
the  Abstraction-Decomposition  map  points  to  information  (either  directly  or  indirectly)  that  must 
be  provided  within  the  workspace,  although  different  stake-holders  (staff  members,  operators) 
will  need  access  to  different  constellations  of  that  information.  This  information  will  reveal  to 
the  workers  the  essential  functional  properties  (purposes,  values,  resources  and  opportunities)  of 
their  work  area.  However  work  remains  beyond  the  development  of  the  Abstraction- 
Decomposition  map  to  generate  the  required  design  specifications. 

AN  EMPHASIS  ON  SOCIO-TECHNICAL  SYSTEMS 

Many  who  undertake  Work  Domain  Analysis  focus  on  the  technical  aspects  of  the  systems  they 
analyze  but  the  central  concern  is  with  socio-technical  systems  and  so  the  value  of  the  analysis  is 
limited  to  the  extent  that  we  do  not  consider  the  social  aspects  of  the  system.  Figure  A-6 
contrasts  a  predominantly  technical  with  a  predominantly  socio-technical  analysis.  An 
instruction  from  a  procedures  manual  fur  librarians  was  used  to  develop  the  abstraction  hierarchy 
in  the  right  panel: 
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When  a  book  is  returned,  draw  a  line  with  a  black  Magic  marker  through  the 
name  to  protect  the  privacy  of  the  borrower,  replace  the  card  in  the  book  and  then 
return  the  book  to  the  shelf. 

That  abstraction  hierarchy  was  developed  as  a  tutorial  exercise  in  making  the  design  rationale  for 
this  instruction  explicit.  Why  a  magic  marker?  Why  black?  These  become  apparent  in  relation  to 
the  privacy  issue  but  other  types  of  markers  may  do  as  well.  Or  are  there  additional  reasons? 

The  general  issue  here  is  that  design  rationale  is  generally  not  made  explicit  in  procedural 
specifications  and  workers  can  adapt  without  realizing  they  are  violating  the  design  rationale. 

The  development  of  an  Abstraction-Decomposition  map  is  a  step  towards  developing  an  explicit 
representation  of  design  rationale  that  can  be  a  made  available  to  librarians  so  that  they  can 
respond  not  necessarily  to  the  technical  meaning  of  this  instruction  but  to  the  intent  behind  it. 


Figure  A-6.  Two  tutorial  examples  of  the  Abstraction-Decomposition  representation,  a 
primarily  technical  system  (Home  Cooling,  left  panel)  and  a  socio-technical  system 

(Library  Client  Tracking,  right  panel) 


SOCIO-TECHNICAL  SYSTEMS  HAVE  STRANGE  PROPERTIES 

The  functional  properties  of  engineered  systems  are  designed  to  be  stationary.  Non-stationarity 
intrudes  when  parts  wear  out  but  in  the  main,  things  stay  the  same.  Such  is  not  the  case  when  we 
insert  humans  into  the  system.  Humans  change  in  themselves  (they  learn,  they  develop,  they 
age)  and  they  frequently  modify  the  functionality  of  the  systems  they  use.  Many  who  undertake 
Work  Domain  Analysis  focus  on  the  technical  aspects  of  the  systems  they  analyze  but  there  is 
added  value  in  laying  out  those  functional  properties  that  are  non-stationary  because  of  human 
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participation.  Unlike  the  non-stationarity  of  technical  systems,  human-induced  non-stationarity 
will  often  enhance  system  effectiveness  and  should  be  promoted  rather  than  avoided. 

The  contrast  of  a  predominantly  technical  versus  socio-technical  analysis  as  shown  in  Figure  A-7 
and  Figure  A-8  was  developed  to  illustrate  this  point.  The  IPOD  example  is  characteristic  of 
many  of  the  analyses  found  in  the  literature  but  the  theatrical  example  illustrates  a  challenging 
feature  of  socio-technical  systems  that  is  often  neglected.  The  functionality  of  the  parts  can 
change  as  the  system  evolves.  The  director  might  take  on  acting  functions  and  actors  might  take 
on  directing  responsibilities.  In  addition,  actors  might  develop  and  adapt  their  capabilities.  For 
example,  a  comedy  specialist  might  develop  as  a  dramatic  actor  and  over  the  life  of  an  extended 
production  might  begin  to  inject  dramatic  elements  into  the  performance.  How  does  that  change 
the  system;  in  this  case  the  theatrical  experience  for  audience,  performers  and  director? 
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Figure  A-7.  A  predominantly  technical  analysis  of  an  IPOD 
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Figure  A-8.  A  socio-technical  analysis  of  a  theatrical  production 


WHY  ABSTRACTION-DECOMPOSITION? 

Why  might  we  believe  that  an  Abstraction-Decomposition  map  is  a  useful  form  of 
representation?  Jens  Rasmussen  has  shown  that  expert  trouble-shooters  and  expert  problem- 
solvers  navigate  through  an  Abstraction-Decomposition  space  much  like  that  represented  in  an 
Abstraction-Decomposition  map  as  the  solve  problems.  Typically,  they  start  with  purposes  or 
values  at  the  system  level  and  then  work  down  towards  decompositions  at  physical  object  and 
physical  process  levels.  Also,  typically,  the  trajectory  is  irregular,  opportunistic  and  iterative. 

This  pattern  is  depicted  in  Figure  A-9  and  Figure  A-10  (from  Hoffman  and  Lintem,  2006)  where 
a  fragment  of  a  work  domain  for  weather  forecasters  (Figure  A-9)  is  overlaid  with  a  scenario 
trajectory  (Figure  A-10).  The  numbers  race  the  sequence  in  which  one  subject  matter  expert 
visited  the  nodes  of  the  Abstraction-Decomposition  map  were  within  this  scenario.  The  callouts 
reveal  the  comments  made  by  the  subject  matter  expert  at  each  node. 
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Figure  A-9.  An  abstraction-decomposition  matrix  of  a  fragment  of  a  weather  forecasting 

work  domain 

(From  Hoffman  and  Lintern,  2006;  Figure  12.2) 


Figure  A-10.  An  abstraction-decomposition  matrix  of  a  fragment  of  a  weather  forecasting 
work  domain  with  an  activity  overlay  (statements  in  callouts  are  quotes  from  a  forecaster) 
(From  Hoffman  and  Lintern,  2006;  Figure  12.3) 


63 


The  claim  is  that  we  all  do  that  (at  least  implicitly)  every  time  we  solve  a  moderately  complex 
problem.  The  commitment  to  the  Abstraction-Decomposition  space  constitutes  a  theory  of 
cognition,  albeit  one  that  has  not  yet  been  enunciated  explicitly  or  in  detail.  Can  you  believe  the 
claim  that  we  navigate  through  an  Abstraction-Decomposition  space  when  we  plan  or  solve 
problems?  If  you  cannot,  you  probably  should  not  build  Abstraction-Decomposition  maps.  On 
the  other  hand,  if  you  can  believe  that  this  is  fundamental  to  the  way  people  behave,  you  should 
find  Work  Domain  Analysis  to  be  a  useful  exercise.  Furthermore,  your  understanding  of  that 
theory  is  your  best  guide  to  how  you  construct  the  Abstraction-Decomposition  map;  what  the 
levels  of  abstraction  mean,  what  sort  of  concepts  go  into  each  of  those  levels,  and  how  you  deal 
with  decomposition. 

Also  remember  that  the  Abstraction-Decomposition  map  is  not  a  formal  system  as  is,  for 
example,  mathematics.  The  elements  of  a  formal  system  must  be  defined  precisely  and  the 
relationships  between  them  must  be  logical  and  consistent.  Many  of  the  critiques  in  the  literature 
point  to  logical  inconsistencies  in  the  way  that  the  various  relationships  of  the  Abstraction- 
Decomposition  map  are  defined  and  used,  but  remember  that  this  was  never  intended  to  be  a 
formal  system.  It  was  always  intended  to  be  a  depiction  of  the  way  subject  matter  experts  could 
think  effectively  about  their  work  domain.  Many  appear  to  believe  that  thinking  processes 
correspond  to  formal  computational  processes  but  the  underlying  assumption  of  Work  Domain 
Analysis  is  that  the  most  effective  forms  of  thinking  are  irregular,  opportunistic  and  iterative; 
they  are  not  irrational  but  they  are  very  far  from  anything  like  a  formal  computational  process. 

THE  RELATIONSHIP  TO  SYSTEMS  ENGINEERING 

Some  Systems  Engineers  dispute  the  contribution  of  a  Work  Domain  Analysis.  They  claim  that 
this  is  a  Systems  Engineering  tool  that  is  already  discussed  adequately  in  standard  textbooks. 
However,  other  Systems  Engineers  state  that  nothing  like  Work  Domain  Analysis  exists  within 
their  discipline. 

Decomposition  is  used  extensively,  systematically  and  explicitly  within  Systems  Engineering 
(e.g.,  Blanchard  and  Fabrycky,  1990).  In  contrast,  the  commitment  to  functional  abstraction  is 
less  clear.  Tools  such  as  Attributes  Lists  (which  organize  system  features  into  the  three  broad 
categories  of  objectives,  constraints  and  functionality),  Hierarchal  Objective  Lists  (which  can  be 
configured  into  Objective  Trees)  and  Morphological  Charts  (also  referred  to  as  Function-Means 
Charts  and  Concept  Combination  Tables)  are  activity-independent  analyses  that  use  dimensions 
of  classification  somewhat  like  the  abstraction  dimension  of  Work  Domain  Analysis. 
Nevertheless,  it  is  not  clear  that  these  tools  are  used  widely  or  that  their  products  are  well 
integrated  into  the  Systems  Engineering  process.  In  addition,  although  Systems  Engineering  texts 
refer  to  purposes  and  values,  they  do  not  clarify  how  those  purposes  and  values  should  be 
integrated  into  the  design  of  a  system  or  how  they  might  be  implemented  as  constraints  on  design 
and  use. 

The  magnitude  of  the  intellectual  debt  owed  Systems  Engineering  by  Cognitive  Work  Analysis 
remains  unclear,  but  of  more  concern  is  whether  the  products  of  Cognitive  Work  Analysis  can  be 
of  value  within  the  broad  scope  of  the  systems  design  process.  The  design  of  complex  socio- 
technical  systems  continues  to  pose  significant  challenges,  one  of  which  is  the  effective 
deployment  and  use  of  human  resources.  Anything  we  can  do  as  cognitive  engineers  to  resolve 
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those  challenges  will  be  of  considerable  benefit  and  the  fact  that  the  major  concepts  of  Work 
Domain  Analysis  are  already  familiar  to  Systems  Engineers  is  a  point  of  contact  between  the  two 
disciplines  that  should  support  rather  than  detract  from  this  effort. 
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APPENDIX  B.  WHAT  DO  THE  ABSTRACTION  LABELS  MEAN? 
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BACKGROUND 


This  discussion  of  abstraction  labels  was  developed  by  Dr.  Gavan  Lintem.  Dr.  Lintem  serves 
Chief  Scientist  at  General  Dynamics  -  Advanced  Information  Engineering  Services.  Dr.  Lintem 
is  a  Subject  Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering.  Ele  previously  served 
as  Director  of  Human  Factors  for  the  Air  Operations  division  of  Australia’s  Defense  Science  and 
Technology  systems  research  where  he  managed  and  built  a  program  in  Cognitive  Systems 
Engineering. 

SYSTEM  PURPOSE 

The  meaning  of  System  Purpose  is  self-explanatory. 

•  What  do  we  want  to  achieve  with  this  system? 

•  Why  does  the  system  exist?  Why  is  it  being  designed? 

•  Purpose  may  be  multi-dimensional. 

VALUES  &  PRIORITIES 

What  are  the  values  that  shape  how  you  use  this  system,  specifically  how  you  use  it  to  satisfy  the 
purpose?  What  abstract  properties  help  you  establish  priorities  with  respect  to  functional 
purpose?  What  are  your  guiding  concerns? 

•  What  considerations  guide  what  you  do?  Most  significantly,  what  considerations 
constrain  how  you  set  priorities  and  allocate  resources? 

•  Properties  of  balance,  conservation,  preservation,  minimization,  maximization  are 
important,  e.g.  safety-productivity  tradeoff. 

•  Policies  and  Legislation  will  shape  strategies,  e.g.  Rules  of  Engagement,  Geneva 
Convention  (what  is  the  underlying  value?) 

A  good  way  to  view  this;  you  get  the  job  done,  it  works,  all  is  fine,  but  you  still  think  it  is  not  a 
good  job  because  you  did  not  attend  to  certain  details.  The  way  you  have  done  it  is  not  elegant, 
it  does  not  conform  to  the  rules  or  it  was  risky.  Others  may  not  understand  why  you  did  it  that 
way.  You  did  not  guard  against  potential  problems.  It  worked  this  time  but  you  were  lucky. 

PURPOSE-RELATED  FUNCTIONS 

These  are  the  essential  functions  irrespective  of  physical  nature  of  the  system,  e.g., 
communication  with  reference  to  System  Purpose  via  Values  and  Priorities  but  without  reference 
to  the  mode  of  communication. 

•  Plan,  Organize,  Inform,  Maintain,  Produce,  Guide,  Navigate,  Generate,  Communicate, 
Administer,  Serve,  Purchase,  Exhibit,  Mediate 

•  Properties  sufficient  to  identify  work  domain  functions  that  must  be  coordinated, 
regardless  of  how  they  are  physically  implemented,  a  meaningful  combination  of 
properties  of  the  physical  functions 
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PHYSICAL  FUNCTIONS  (&  EFFECTS) 


These  are  the  specific  functions  of  the  physical  elements  of  the  system;  the  properties  necessary 
and  sufficient  for  control  of  physical  work  activities  and  use  of  equipment. 

•  Find.  Retrieve.  Store.  Position.  Use,  Talk.  Signal.  Lift.  Push 

•  The  word  effects  is  added  because  you  might  be  concerned  with  the  effects  of  the 
environment  (e.g.,  weather).  It  sounds  strange  to  say  weather  has  a  function  although, 
conceptually  at  least,  weather  effects  are  similar  to  physical  functions.  Consequently,  at 
this  level,  the  term  function  is  reserved  for  the  consequences  of  engineered  artifacts  while 
the  term  effect  is  used  to  refer  to  the  consequences  of  natural  phenomena  such  as  weather. 

PHYSICAL  PROPERTIES 

Specify  names  of  physical  devices,  colors,  shapes,  locations,  etc.  But  only  introduce  descriptors 
that  are  relevant  to  your  design  purpose.  For  example,  shape  of  an  aircraft  is  not  useful  if  you 
are  designing  a  flight  simulator  for  that  aircraft  but  is  useful  if  you  are  designing  an  aircraft 
identification  system. 

These  are  the  properties  necessary  and  sufficient  for  classification,  identification  and  recognition 
of  particular  material  objects  and  their  configuration,  and  for  navigation  through  the  system. 

TIPS 

Tip  for  Identifying  Physical  Functions  and  Physical  Properties 

•  Ask  your  subject  matter  experts  what  they  would  use  to  satisfy  a  Purpose-Related 
Function  such  as  communication  {How  would  you  communicate  with  your  forward 
observer?).  Some  may  respond  with  an  object  (/  use  the  radio,  the  radar)  or  they  may 
respond  with  a  physical  function  (/  send  a  message  about  this,  1  use  the  location  and 
direction  of  the  target).  To  the  object  response,  you  ask  what  does  it  do?  To  the 
physical  function  response  you  ask,  what  system  gives  you  that?  Record  both  and  link 
them. 

Tip  for  Decomposing 

•  Decompose  to  level  of  possible  action 

Tip  for  the  early  stages  of  system  design 

•  You  might  only  need  to  develop  the  top  three  levels  of  the  abstraction  hierarchy. 
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APPENDIX  C.  REPRESENTATIONAL  ISSUES  AND  REPRESENTIONAL  FORMS 


BACKGROUND 


The  following  commentary  reflects  Dr.  Lintem’s  perspective  of  representational  issues  and 
representational  forms  in  Cognitive  Systems  Engineering.  Dr.  Lintem  serves  Chief  Scientist  at 
General  Dynamics  -  Advanced  Information  Engineering  Services.  Dr.  Lintem  is  a  Subject 
Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering.  He  previously  served  as  Director 
of  Human  Factors  for  the  Air  Operations  division  of  Australia’s  Defense  Science  and 
Technology  systems  research  where  he  managed  and  built  a  program  in  Cognitive  Systems 
Engineering. 


REPRESENTATIONAL  ISSUES 
Issue:  Tightly-Coupled  Control 

It  is  a  major  concern  that  almost  all  representations  used  by  Systems  Engineers  and  Cognitive 
Engineers  imply  tightly  coupled  control.  The  dangers  of  tightly  coupled  control  within  human 
endeavors  are  well  known;  it  is  often  referred  to  as  micro-management  and  is  the  reason  that 
work  unions  can  impose  the  seemingly  odd  tactic  of  a  work-to-regulations  strike  (Lintem,  2003). 
Following  an  exceptional  analysis  of  a  major  system  accident,  Snook  (2000)  recognized  the 
danger  but  apparently  could  not  envision  an  alternative.  Tightly  coupled  control  of  systems  that 
include  humans  as  functional  components  are  prone  to  instability.  Although  that  is  now  well 
known,  many  in  Cognitive  Systems  Engineering — who  will  readily  deny  any  commitment  to 
tightly  coupled  control  in  personal  conversation  (e.g.,  Erik  Hollnagel,  David  Woods,  Kim 
Vicente) — continue  to  make  occasional  statements  in  their  published  work  (Hollnagel  &  Woods, 
1999;  Vicente,  2004)  and  in  their  professional  presentations  that  are  most  readily  taken  to  imply 
that  commitment. 

This  problem  almost  certainly  results  from  the  now  long-established  practice  of  looking  towards 
technology  for  images  of  mechanism  (e.g.  the  computer)  to  support  our  understanding  of  how 
cognitive  systems  work.  In  contrast,  the  single  most  significant  contribution  from  Cognitive 
Engineering  towards  understanding  human  behavior  is  the  radical  move  by  its  founders  (e.g. 
Klein,  1989;  Rasmussen,  1986)  of  looking  towards  the  details  of  human  work  activity  and  human 
work  experience  for  insightful  images.  This  move  is  not  well  recognized  even  among 
practitioners  of  Cognitive  Engineering.  Some,  for  example,  argue  that  interest  in  cognitive 
issues  is  the  defining  contribution  of  Cognitive  Engineering,  and  accompany  that  observation 
with  the  spurious  claim  that  Human  Factors  and  Engineering  Psychology  do  not  deal  with 
cognitive  issues  (e.g.,  Hollnagel  &  Woods,  1999). 

Issue:  Decomposition 

The  focus  in  the  past  has  been  on  the  Cartesian  approach  of  breaking  a  problem 
into  smaller  components,  solving  each  of  the  smaller  problems,  and  then 
integrating  the  pieces  back  into  a  whole  solution. 

International  Council  on  Systems  Engineering  (2005) 

Decomposability  has  been  the  basis  of  standard  engineering  design  practice,  which  in  itself 
provides  one  of  the  challenges  to  Human  Systems  Integration.  The  assumption  that  technical 
systems  can  be  disassembled  into  independent  modules  is  conceptually  safe  because  engineered 
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systems  are  fabricated  from  independent  modules;  essentially  that  has  always  been  the  dominant 
design  strategy.  In  contrast — and  despite  what  one  commonly  sees  in  textbooks  by  cognitive 
psychologists — the  human  cognitive  system  is  not  decomposable;  the  attempt  to  decompose  the 
human  cognitive  system  results  in  modules  that  are  interdependent,  a  fact  often  appreciated  by 
those  who  do  the  decomposition  and  who  attempt  to  deal  with  this  issue  by  the  use  of  a  dense 
network  of  feedback  and  feed-forward  loops.  Unfortunately,  that  representational  strategy 
implies  tight  coupling,  which  is  definitely  not  the  characteristic  mode  of  interaction  within  the 
human  cognitive  system.  Furthermore,  the  insight  that  can  be  derived  from  a  representation  in 
which  modules  represent  arbitrary  decompositions  and  every  node  is  connected  to  every  other 
node  is  limited. 


REPRESENTATIONAL  FORMS 
Abstraction-Decomposition 

Practitioners  of  Cognitive  Work  Analysis  are  occasionally  criticized  for  their  emphasis  on  Work 
Domain  Analysis,  it  often  being  the  only  phase  of  Cognitive  Work  Analysis  that  is  completed. 

In  their  recent  book,  Bums  and  Hajdukiewicz  (2004)  ignore  the  remaining  four  phases  on  their 
way  to  developing  an  approach  to  Ecological  Interface  Design.  Although  this  emphasis  might  be 
seen  as  a  weakness,  it  has  resulted  in  the  relative  maturation  of  Work  Domain  Analysis  and  its 
analytic  product,  the  Abstraction-Decomposition  matrix.  Concerns  remain,  most  notably  in  the 
definition  of  terms  that  are  central  to  the  construction  of  an  Abstraction-Decomposition  matrix, 
but  considerable  value  has  accrued  in  terms  of  product  maturation  from  this  focus  on  the  Work 
Domain  Analysis. 

Decomposition  is  used  extensively,  systematically,  and  explicitly  within  Systems  Engineering 
(e.g.,  Blanchard  and  Fabrycky,  1990).  In  contrast,  the  commitment  to  functional  abstraction  is 
less  clear.  Activity-independent  analyses  that  use  dimensions  of  classification  somewhat  like  the 
abstraction  dimension  of  Work  Domain  Analysis  (e.g.,  Attributes  Lists,  Hierarchal  Objective 
Lists  and  Morphological  Charts)  are  available,  but  it  is  not  clear  that  they  are  widely  used  or  that 
their  products  are  well  integrated  into  the  analytic  processes  of  Systems  Engineering.  For 
example,  the  Department  of  Defense  architectural  framework  does  not  include  a  dimension  of 
abstraction  in  any  of  its  26  different  forms  of  representation  (DOD  Architecture  Framework 
Working  Group,  2004). 

Abstraction  is  a  challenging  concept.  The  potential  contribution  of  the  abstraction  dimension 
may  be  little  appreciated  because  it  is  poorly  understood  and  is  readily  confused  with 
decomposition.  Sarcedoti  (1974)  is  one  who — in  proposing  a  hierarchy  of  abstraction  spaces  as 
a  means  of  reducing  combinatorial  complexity  for  planning — appears  at  first  to  appreciate  the 
contribution  of  an  abstraction  analysis.  However,  having  outlined  this  proposition,  Sarcedoti 
then  proceeds  to  treat  abstraction  in  terms  of  decomposition.  Similarly,  Ayn  Rand  uses 
abstraction  as  a  key  idea  in  support  of  her  Objectivist  Epistemology,  but  she  also  confuses 
abstraction  with  decomposition  (original  papers  by  Ayn  Rand,  republished  in  Binswanger  & 
Peikoff,  1990) 

Indeed,  this  is  an  issue  that  those  of  us  who  develop  Abstraction-Decomposition  matrices  have 
faced.  Almost  without  exception,  we  have  struggled  in  our  early  experiences  with  Work  Domain 
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Analysis  to  resist  the  temptation  to  decompose  along  the  abstraction  dimension,  to  build  what 
would  essentially  amount  to  a  Decomposition-Decomposition  matrix.  The  distinction  between 
abstraction  and  decomposition  is  logical  but  not  obvious,  and  those  who  analyze  complex  socio- 
technical  systems  will  not  turn  their  attention  to  it  unless  encouraged  to  do  so.  This  is  possibly  a 
significant  contribution  that  Cognitive  Engineers  can  make  to  Systems  Engineering  practice. 

Subsumption 

Subsumption  is  a  hierarchical  structure  in  which  activities  at  a  subordinate  level  are  subsumed 
under  a  super-ordinate  activity.  The  relationship  between  the  super-ordinate  and  subordinate 
activities  is  one  of  supervisory  management;  the  super-ordinate  activity  initiates,  monitors  and 
terminates  the  subordinate  activity — but  beyond  that,  the  subordinate  node  is  autonomous 
(management,  not  control). 

•  The  subsumption  architecture  may  extend  over  several  levels. 

•  Adjacent  levels  require  two-way  communication;  supervisory  management  flows  from 
super-ordinate  to  subordinate,  and  status  updates  and  product  delivery  flow  from 
subordinate  to  the  super-ordinate. 

•  One  super-ordinate  node  may  manage  many  subordinate  nodes. 

A  representational  form  based  on  subsumption  architecture  captures  many  noteworthy  aspects  of 
cognitive  work  within  a  complex,  socio-technical  context,  and  does  so  more  coherently  than  does 
a  representational  form  based  on  the  ubiquitous  linear-flow  model.  A  subsumption 
representation  implies  a  coordinated  coalition.  The  separate  activities  are  initiated  as  needed — 
sometimes  simultaneously  and  sometimes  overlapping.  Different  activities  may  sometimes  be 
active,  then  inactive,  and  then  active  again,  as  managed  at  the  supervisory  level. 


Figure  C-l.  A  three-level  subsumption  representation  of  Time  Sensitive  Target  planning  in 
which  background  activities  are  interrupted  by  the  arrival  of  a  high  priority  task  and 

resumed  when  that  task  is  completed. 

Subsumption  depicts  workflow  as  a  composition  (versus  a  causal  sequence)  of  behaviors  acting 
independently  but  orchestrated  as  a  coherent  performance  on  the  basis  of  supervisory  priorities. 
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It  permits  a  meaningful  representation  of  distributed  supervisory  management  and  can  capture 
essential  elements  of  both  individual  and  team  (distributed)  behavior  as  in: 

•  Activities  remain  coherent  even  if  the  context  changes;  emerging  and  changing  needs  can 
be  accommodated. 

•  Supervisory  management  (versus  micromanagement);  how  management  can 
communicate  with  subordinate  entities  and  how  it  can  coordinate  their  efforts  (an  issue 
for  all  distributed  cognition  and  the  essence  of  teamwork)  so  that  those  who  execute  are 
given  the  freedom  to  exploit  their  strengths  as  they  contribute  to  the  common  goal. 

•  Interrupt  and  resume  activities,  which  are  ubiquitous  within  socio-technical  work,  are 
handled  conveniently  by  having  all  subordinate  nodes  active  and  in  competition  for  time 
so  that  a  super-ordinate  node  may  suspend  or  abort  an  active  subordinate  node  and 
activate  another  to  simulate  adaptive  or  flexible  adjustment  to  changes  in  context  or 
knowledge  (Figure  C-l). 
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APPENDIX  D.  THE  RELATIONSHIP  OF  COGNITIVE  ENGINEERING  TO  SYSTEMS 

ENGINEERING 


Background 

This  discussion  was  developed  by  Dr.  Gavan  Lintem.  Dr.  Lintem  serves  Chief  Scientist  at 
General  Dynamics  -  Advanced  Information  Engineering  Services.  Dr.  Lintem  is  a  Subject 
Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering.  He  previously  served  as  Director 
of  Human  Factors  for  the  Air  Operations  division  of  Australia’s  Defense  Science  and 
Technology  systems  research  where  he  managed  and  built  a  program  in  Cognitive  Systems 
Engineering. 


Discussion 

Some  Systems  Engineers  dispute  the  contribution  of  a  Work  Domain  Analysis.  They  claim  that 
this  is  a  Systems  Engineering  tool  that  is  already  discussed  adequately  in  standard  textbooks. 
However,  other  Systems  Engineers  state  that  nothing  like  Work  Domain  Analysis  exists  within 
their  discipline. 

Decomposition  is  used  extensively,  systematically  and  explicitly  within  Systems  Engineering 
(e.g.,  Blanchard  and  Fabrycky,  1990)9.  In  contrast,  the  commitment  to  functional  abstraction  is 
less  clear.  Tools  such  as  Attributes  Lists  (which  organize  system  features  into  the  three  broad 
categories  of  objectives,  constraints,  and  functionality),  Hierarchal  Objective  Lists  (which  can  be 
configured  into  Objective  Trees)  and  Morphological  Charts  (also  referred  to  as  Function-Means 
Charts  and  Concept  Combination  Tables)  are  activity-independent  analyses  that  use  dimensions 
of  classification  somewhat  like  the  abstraction  dimension  of  Work  Domain  Analysis. 
Nevertheless,  it  is  not  clear  that  these  tools  are  used  widely  or  that  their  products  are  well 
integrated  into  the  Systems  Engineering  process.  In  addition,  although  Systems  Engineering 
texts  refer  to  purposes  and  values,  they  do  not  clarify  how  those  purposes  and  values  should  be 
integrated  into  the  design  of  a  system  or  how  they  might  be  implemented  as  constraints  on  design 
and  use. 

The  magnitude  of  the  intellectual  debt  owed  Systems  Engineering  by  Cognitive  Work  Analysis 
remains  unclear,  but  of  more  concern  is  whether  the  products  of  Cognitive  Work  Analysis  can  be 
of  value  within  the  broad  scope  of  the  systems  design  process.  The  design  of  complex  socio- 
technical  systems  continues  to  pose  significant  challenges — one  of  which  is  the  effective 
deployment  and  use  of  human  resources.  Anything  we  can  do  as  cognitive  engineers  to  resolve 
those  challenges  will  be  of  considerable  benefit  and  the  fact  that  the  major  concepts  of  Work 
Domain  Analysis  are  already  familiar  to  Systems  Engineers  is  a  point  of  contact  between  the  two 
disciplines  that  should  support  rather  than  detract  from  this  effort. 


9  Blanchard,  B.  S.,  &  Fabrycky,  W.  J.  (1990).  Systems  Engineering  and  Analysis  (2nd  ed.).  Englewood  Cliffs,  NJ: 
Prentice  Hall. 
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APPENDIX  E.  A  DISCUSSION  WITH  DR.  GUL  KREMER  REGARDING  HOW 
COGNITIVE  ENGINEERS  MIGHT  SUPPORT  SYSTEMS  ENGINEERS 
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BACKGROUND 


The  discussion  reported  below  took  place  between  Dr.  Lintem  and  Dr.  Kremer  on 
13  March  2005  at  Pennsylvania  State  University. 

Dr.  Lintem  serves  Chief  Scientist  at  General  Dynamics  -  Advanced  Information  Engineering 
Services.  Dr.  Lintem  is  a  Subject  Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering. 
He  previously  served  as  Director  of  Human  Factors  for  the  Air  Operations  division  of  Australia’s 
Defense  Science  and  Technology  systems  research  where  he  managed  and  built  a  program  in 
Cognitive  Systems  Engineering. 

Dr.  Gul  Kremer  is  an  Assistant  Professor  of  Engineering  Design,  Pennsylvania  State  University. 
She  was  a  Summer  Faculty  Fellow  at  the  Air  Force  Research  Laboratory’s  Human  Effectiveness 
Directorate  in  2002,  2003,  2004,  and  2005. 

DISCUSSION 

What  are  the  important  cognitive  issues? 

This  is  not  self-evident  and  many  of  the  cognitive  issues  on  which  people  focus,  such  as  those 
relating  to  interface  design,  are  not  the  most  critical.  We  should  be  addressing  those  that  face 
systems  engineers  early  in  the  design  phase,  particularly  in  the  concept  definition  phase,  for 
example: 

•  How  might  we  incorporate  load  issues  relating  to  fatigue?  How  long  can  people  stay  on 
the  job?  What  are  the  handover  costs? 

•  How  do  we  make  resource  assignments? 

•  How  can  we  automate  support  decisions  and  what  interface  should  be  used? 

•  What  level  of  operator  intervention  should  we  achieve — the  operator  has  to  enter  into  the 
decision  points  at  critical  gates — so  what  are  the  critical  decision  points? 

•  Visualization — operators  emphasize  the  common  operating  picture — can  we  develop  a 
visualization  that  will  help  them  make  their  decisions? 

We  discussed  the  need  for  a  design  case  study  that  will  put  together  the  design  requirements  to 
realize  operational  needs.  The  purpose  would  be  to  allow  examination  of  what-if  scenarios  to 
reveal  bottlenecks. 


Workload 

How  might  we  reveal  to  a  systems  engineer  how  operator  workload  would  affect  a  proposed 
design?  For  physical  issues,  there  are  tables  to  show  how  well  people  can  work  under  load. 
Systems  engineers  are  fond  of  this  sort  of  thing  and  one  thought  we  explored  is  that  we  might 
develop  nomograms  for  workload  or  time  demands  of  a  task  or  activity.  Nevertheless,  this 
would  appear  to  be  an  optimistic  project.  There  is,  as  yet,  no  standard  way  of  measuring 
workload  and  even  if  we  were  to  settle  on  a  standard  procedure,  we  will  still  often  need  to  assess 
workload  for  systems  for  which  there  is  no  prototype  on  which  the  tasks  can  be  examined. 
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Time,  when  used  to  assess  performance,  does  not  have  the  same  measurement  problem  as 
workload — but  again,  assessment  of  the  demand  (how  long  an  activity  will  take  to  perform)  is  a 
challenge.  As  an  example  of  a  problem,  consider  the  activity  of  original  cataloging  within  a 
library.  Cataloging  comes  in  two  basic  forms.  Copy  cataloging  is  the  process  of  taking  an 
existing  record  and  adapting  it  for  the  specific  library's  purpose.  Original  cataloging  is  more 
demanding.  It  requires  that  the  cataloguer  generate  an  entirely  new  record.  The  cataloguer 
creates  that  record  based  on  the  understanding  gleaned  from  examination  of  the  source 
document.  The  creation  of  an  original  record  can  take  anywhere  from  several  minutes  to  several 
hours.  The  huge  difference  in  time  results  because  of  the  following  factors: 

•  the  difficulty  and  clarity  of  the  source  material 

•  the  experience  of  the  cataloguer  both  in  terms  of  general  experience  and  in  terms  of 
specific  experience  with  the  domain  in  question 

•  the  stated  requirements  for  specificity  and  detail — research  libraries,  for  example,  require 
more  specific  and  detailed  records  then  do  public  libraries 

•  the  response  speed  of  the  cataloging  system,  which  may  vary  from  day-to-day  and 
throughout  the  day  (most  cataloging  is  done  on  networked  systems,  many  of  which  are 
shared  by  a  network  of  libraries) 

Probably  the  only  way  to  anticipate  how  long  something  like  original  cataloging  would  take  to 
accomplish  would  be  to  decompose  the  activity  into  sub-tasks  and  then  develop  a  model  that 
would  take  into  account  the  various  contingencies  and  complexities. 

What  are  we  trying  to  achieve? 

Almost  all  of  the  discussions  about  how  Cognitive  Engineers  might  support  Systems  Engineers 
take,  as  a  starting  point,  the  need  to  represent  cognitive  knowledge  in  a  form  that  Systems 
Engineers  can  understand  and  also  integrate  within  their  own  style  of  developing  system  design 
specifications.  I  wish  to  propose  a  different  perspective  that  we  might  explore  over  the  coming 
months;  that  we  should  be  developing  simulation  environments  that  permit  Systems  Engineers  to 
explore  the  cognitive  ramifications  of  their  design  solutions.  My  motivation  for  proposing  this 
alternate  approach  is  that  I  believe  it  is  unrealistic  to  think  that  we  can  specify  cognitive 
parameters  with  precision  sufficient  to  aid  the  design  effort  of  Systems  Engineers. 

Consider  the  possibility  that  the  biggest  challenge  facing  systems  engineers  as  they  formulate  a 
system  concept  is  that  they  know  very  little  about  work  structure  or  process.  The  information 
that  might  help  them  develop  an  effective  system  might  be  gained  from  extensive  experience 
within  a  working  system  prototype.  The  regularly-scheduled  tests  of  Air  and  Space  Operations 
Center  processes  (Joint  Expeditionary  Force  Experiments  or  JEFX)  might  offer  such 
opportunities,  but  are  hugely  expensive  and  difficult  to  conduct.  In  addition,  that  system  is  far 
too  complex.  Simple  human-in-the-loop  simulations,  such  as  the  procedure  of  tabletop  analysis 
used  by  Naikar,  Pearce,  Drumm  and  Sanderson  (2003)'°  would  also  appear  to  offer  an 
opportunity,  but  this  sort  of  analysis  requires  technical  skills  that  systems  engineers  typically  do 
not  have  (see  Appendix  F  for  a  discussion  of  tabletop  analysis). 


10  Naikar,  N.,  Pearce,  B.,  Drumm,  D.  &  Sanderson,  P.  M.  (2003).  Technique  for  designing  teams  for  first-of-a-kind 
complex  systems  with  cognitive  work  analysis:  Case  study.  Human  Factors,  45(2),  202-217. 
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I  propose  that  the  solution  lies  in  the  development  of  an  appropriate  dynamic  simulation  process 
model  for  the  system  under  consideration.  The  model  would  essentially  replicate  the  process  of 
tabletop  analysis  as  used  by  Naikar  et  al.  (2003).  The  content  of  the  model  would  still  have  to  be 
developed  via  a  procedure  like  tabletop  analysis  and  so  would  require  an  intensive  effort  by  a 
suitably  trained  Cognitive  Engineer,  but  once  developed  it  could  be  used  by  others  to  explore  the 
parameter  space.  This  offers  the  significant  advantage  of  being  the  type  of  tool  with  which 
Systems  Engineers  would  typically  be  comfortable.  In  addition,  once  the  time-consuming 
process  of  model  development  has  been  completed,  many  parametric  variations  could  be 
examined  with  much  less  effort.  Such  is  not  the  case  with  tabletop  analysis. 

I  wish  to  stress  at  this  point  in  that  the  proposed  modeling  effort  is  not  one  that  is  aimed  at 
identifying  cognitive  parameters — but  is  aimed  specifically  at  providing  the  Systems  Engineer  as 
designer  with  the  sort  of  experience  that  will  promote  sensitivity  to  and  appreciation  of  the 
important  cognitive  issues  that  have  to  be  considered  in  design.  This  purpose  is  consistent  with 
the  use  of  tabletop  analysis  by  Naikar  et  al.  (2003).  Although  parameter  specification  might 
seem  a  desirable  goal,  I  currently  view  it  as  an  unrealistic  one.  On  the  other  hand,  it  does  seem 
realistic  that  we  could  help  a  designer  develop  an  appreciation  of  the  significance  of  parametric 
variations  within  different  dimensions,  and  I  propose  that  such  a  result  would  generate  a  far  more 
robust  design  strategy  than  is  the  case  currently. 

Representational  Forms 

An  assumption  of  this  project  is  that  the  product  of  any  cognitive  analysis  should  be  provided  as 
a  representation.  Nevertheless,  the  appropriate  form  of  representation  is  in  question.  A 
preliminary  example  of  one  representational  form  is  shown  here  as  Figure  E-l .  To  this  point,  it 
shows  few  details,  but  the  idea  here  is  that  this  is  the  sort  of  form  that  would  be  desirable  as  the 
output  of  a  human-in-the-loop  tabletop  analysis  or  of  a  computer  model.  Ideally,  a  computer 
model  would  show  the  progress  of  activity  in  a  dynamic,  unfolding  fashion  and  would  depict 
communication  events  in  a  time  sequence.  Figure  E-l  shows  a  node  structure  only  for  the 
human  agents  and  would  therefore  reveal  only  the  human-human  interaction.  A  complementary 
representation  of  a  node  structure  for  system  functions  is  also  necessary  to  depict  the  human- 
technology  interactions. 
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APPENDIX  F.  DR.  NEELAM  NAIKAR’S  APPROACH  TO  TABLETOP  ANALYSIS 


Background 

The  discussion  notes  reported  below  are  from  Dr.  Lintem’s  interview  with  Dr.  Naikar  regarding 
tabletop  analysis.  This  discussion  took  place  at  the  Australian  Defence  Science  and  Technology 
Organisation  on  25  January  2005. 

Dr.  Lintem  serves  Chief  Scientist  at  General  Dynamics  -  Advanced  Information  Engineering 
Services.  Dr.  Lintem  is  a  Subject  Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering. 
He  previously  served  as  Director  of  Human  Factors  for  the  Air  Operations  division  of  Australia’s 
Defence  Science  and  Technology  systems  research  where  he  managed  and  built  a  program  in 
Cognitive  Systems  Engineering. 

Dr.  Neelam  Naikar  is  a  Cognitive  Engineer  with  the  Defence  Science  and  Technology 
Organisation  (DSTO),  Melbourne,  Australia. 

Discussion 

Tabletop  analysis  starts  with  a  design  scenario,  developed  through  discussion  with  subject  matter 
experts  that  captures  the  extremes  of  work  intensity  in  terms  of  cognitive  demands  and  workload 
(something  we  termed  “edge  cases”.  The  design  scenario  identifies  the  representative  and 
prototypical  features  of  a  coalition  environment.  Importantly,  it  identifies  prototypical  features, 
typical  geography,  and  timelines  for  work  activities. 

Subject  matter  experts  are  sat  before  a  map  over  a  relevant  area,  with  all  entities  located  at  a 
specific  time.  They  are  given  a  brief  technical  summary  describing  key  events  at  that  time.  The 
researchers  and  the  subject  matter  experts  then  discuss  the  unfolding  of  events  of  the  scenario 
over  successive  time  periods,  relocating  entities  as  necessary  and  talking  about  the  activities  as 
they  relate  to  the  events. 

There  are  two  subject  matter  experts  per  scenario  analysis.  They  are  asked  about  how  they 
would  allocate  work  to  crewmembers  and  what  crew  concept  they  would  like  to  use  as  they 
move  through  the  time  periods  of  the  scenario. 

Neelam  stressed  that  the  purpose  was  not  to  evaluate  crew  concepts  or  performance  of  crews  in 
various  configurations,  but  was  more  for  generating  a  debate  that  would  clarify  the  important 
dimensions  or  properties  of  teams.  Thus,  tabletop  analysis  is  not  about  testing  team  design  but 
about  exploring  the  issues  (e.g.,  if  we  have  a  multi-skilled  team  that  reconfigures  for  different 
challenges,  the  mission  commander  needs  to  devote  resources  towards  management). 

From  this  notion,  Neelam  and  I  discussed  the  sort  that  cognitive  representations  to  be  designed  to 
help  systems  engineers  with  cognitive  demands  problems.  These  representations  should  not 
necessarily  be  aimed  at  specifying  the  cognitive  demands  and  the  design  solutions,  but  should  be 
aimed  at  supporting  a  dialog  between  cognitive  engineers  and  systems  engineers  as  they  seek  to 
resolve  design  issues  surrounding  cognitive  requirements. 


81 


APPENDIX  G.  COMMENTARY  ON  DR.  LIND’S  CRITICAL  ANALYSIS  OF 
ABSTRACTION-DECOMPOSITION  ANALYSES 
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BACKGROUND 


The  following  is  Dr.  Lintem’s  commentary  on  a  critical  analysis  of  Abstraction-Decomposition 
analyses  published  by  Dr.  Lind  in  2003. 

Dr.  Lintem  serves  Chief  Scientist  at  General  Dynamics  -  Advanced  Information  Engineering 
Services.  Dr.  Lintem  is  a  Subject  Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering. 
He  previously  served  as  Director  of  Human  Factors  for  the  Air  Operations  division  of  Australia’s 
Defense  Science  and  Technology  systems  research  where  he  managed  and  built  a  program  in 
Cognitive  Systems  Engineering. 

Dr.  Morten  Lind  is  a  Full  Professor  of  Control  Systems  Engineering  at  the  Technical  University 
of  Denmark,  Lyngby,  Denmark.  His  research  interests  include  human-machine  systems  for 
supervisory  control  and  modeling  of  complex  socio-technical  systems.  From  the  late  1960s  to 
the  early  1980s,  he  served  as  a  Research  Scientist  at  the  Riso  National  Laboratory,  Roskilde, 
Denmark.  He  later  consulted  for  the  Riso  National  Laboratory’s  Cognitive  Systems  Group  in 
1994. 


ABSTRACT 

Lind  (2003)  has  authored  a  critical  analysis  of  the  Abstraction-Decomposition  analysis 
undertaken  in  Cognitive  Work  Analysis.  I  review  his  critique  and  conclude  that  it  is  misguided 
in  many  aspects.  In  my  analysis,  I  touch  on  issues  related  to  Multilevel  Flow  Modeling  and 
Applied  Cognitive  Work  Analysis.  Abstraction-Decomposition  analysis  has  a  unique  role  to 
play  within  Cognitive  Engineering.  Although  only  some  of  the  issues  raised  by  Lind  require 
resolution,  consideration  of  those  selected  issues  would  be  useful  for  the  development  of 
Cognitive  Work  Analysis. 

INTRODUCTION 

Morten  Lind  was  involved  with  Jens  Rasmussen11  in  the  early  developments  of  Cognitive  Work 
Analysis,  but  became  disenchanted  at  least  with  Work  Domain  Analysis.  He  subsequently 
developed  Multilevel  Flow  Modeling  to  address  the  issues  he  sees  as  important  for  Human- 
Systems  analysis.  Multilevel  Flow  Modeling  retains  some  elements  of  an  Abstraction- 
Decomposition  analysis  but  differs  from  it  in  substantive  ways. 

Lind  (2003)  raises  several  challenges  for  Work  Domain  Analysis.  I  review  those  challenges  to 
assess: 

•  whether  his  critique  for  the  Abstraction-Decomposition  format  developed  within  Work 
Domain  Analysis  has  value,  and 

•  whether  we  should  be  taking  notice  of  Multilevel  Flow  Modeling. 


1 1  Jens  Rasmussen  was  a  professor  of  Cognitive  Systems  Engineering  at  the  Riso  National  Laboratory  and  the 
Technical  University  of  Copenhagen.  He  has  conducted  research  in  the  areas  of  human  reliability,  work  domain 
taxonomy,  human-system  integration,  and  ecological  information  systems  design.  He  currently  consults  in 
Cognitive  Systems  Engineering. 
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I  have  reviewed  four  of  Lind’s  papers  and  have  engaged  in  an  e-mail  exchange  with  him.  This 
note  summarizes  my  conclusions.  In  addition,  a  small  number  of  the  arguments  found  in  papers 
that  describe  Applied  Cognitive  Work  Analysis  (e.g.  Elm,  Roth,  Potter,  Gualtieri  &  Easter,  2005) 
are  similar  to  those  forwarded  by  Lind  and  there  is  some  similarity  between  Multilevel  Flow 
Modeling  and  the  Functional  Abstraction  Network  of  Applied  Cognitive  Work  Analysis.  I  note 
that  where  it  is  relevant. 


Overview 

Work  Domain  Analysis  is  a  phase  of  Cognitive  Work  Analysis,  which  in  turn  is  a  framework 
within  the  larger  enterprise  of  Cognitive  Systems  Engineering.  I  have  begun  to  prefer  the  term 
Abstraction-Decomposition  analysis  for  Work  Domain  Analysis  because  the  Work  Domain 
Analysis  normally  undertaken  within  Cognitive  Work  Analysis  is  only  one  of  potentially  many 
ways  to  analyze  the  work  domain.  Bums  and  Vicente  (2001)  note,  for  example,  that  Multilevel 
Flow  Modeling  is  a  form  of  Work  Domain  Analysis. 

Lind  has  emerged  as  notable  critic  of  Work  Domain  Analysis  as  practiced  within  the  Cognitive 
Work  Analysis  framework,  primarily  directing  his  critique  at  the  Abstraction-Decomposition 
format  and  the  means-ends  connections  between  levels  of  abstraction.  In  summary,  he  argues 
that  an  Abstraction-Decomposition  analysis  is  incoherent  and  cannot  perform  the  role  promoted 
for  it  by  Rasmussen,  Vicente,  and  many  others  (including  me),  either  in  principle  or  in  practice. 

Lind  is  not  alone  in  voicing  his  disapproval  of  the  Abstraction-Decomposition  analysis  but,  in 
contrast  to  many  others  whose  critiques  are  little  more  than  expressions  of  discontent,  he  has 
developed  an  argument  with  content.  His  arguments  are  sufficiently  cogent  to  be  addressed  and, 
given  that  they  are  devastating  if  valid,  need  to  be  addressed  by  those  of  us  who  practice  Work 
Domain  Analysis  in  the  form  espoused  by  Rasmussen  and  Vicente.  Lind’s  arguments  are  not 
relevant  to  Applied  Cognitive  Work  Analysis  (Elm,  et  al.,  2005),  which  does  not  lead  to  an 
Abstraction-Decomposition  map  and  which,  in  fact,  leads  to  a  representation  similar  to  that 
produced  in  Multilevel  Flow  Modeling.  Elm,  Potter,  Gualtieri,  Roth  and  Easter  (2003) 
acknowledge  their  intellectual  debt  to  Lind. 

Lind  presents  his  most  comprehensive  critique  in  his  2003  paper,  which  is  essentially  a 
development  of  an  earlier  symposium  paper  (Lind,  1999a).  He  argues  that  the  Abstraction- 
Decomposition  analysis  suffers  from  both  methodological  and  conceptual  problems — 
specifically  that  the  meaning  of  the  abstraction  levels  and  the  means-ends  relations  between  them 
are  not  well  defined  and  that  there  is  no  rationale  for  a  fixed  number  of  abstraction  levels.  He 
concludes  (essentially  without  justification)  that  clarification  of  the  semantics  of  the  abstraction 
hierarchy  will  invariably  reduce  the  range  of  work  domains  to  which  it  can  be  applied.  There  is 
far  too  much  in  Lind’s  2003  paper  to  deal  with  everything,  but  in  what  follows,  I  will  address 
what  I  see  as  the  most  challenging  issues. 

ASSESSMENT  STRATEGY 

The  best  criterion  for  assessing  the  effectiveness  of  an  analytic  method  is  its  effectiveness  in 
supporting  the  project  in  which  it  is  being  used.  That  is  generally  difficult  because  the  results  of 
analysis  are  often  not  transformed  into  design  and  at  other  times,  the  link  between  a  design  and 
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the  preliminary  analysis  remains  obscure.  Assessment  of  an  analytic  method  is  particularly 
troublesome  when  it  is  used  predominantly  in  the  design  of  systems  that  are  to  be  fielded  at  some 
considerable  time  into  the  future.  In  that  case,  analysis  must  proceed  in  the  absence  of 
operational  feedback — and  even  after  a  system  is  fielded  it  may  be  difficult  to  connect 
operational  success  or  failure  to  the  design  techniques  used  in  development. 

Where  analysis  is  directed  at  future  systems,  a  critique  of  the  principles  and  structure  of  the 
analytic  method  can  be  useful  if  it  reveals  one  or  more  of  the  following  problems,  listed  in  order 
from  the  most  to  least  serious: 

•  The  method  is  poorly  motivated  and  has  nothing  to  offer  even  if  done  well, 

•  The  basic  principles  are  fatally  flawed  and  although  the  method  is  well  motivated  and 
without  a  challenger,  it  cannot  accomplish  anything  useful, 

•  There  is  a  different  analytic  strategy  that  will  accomplish  what  that  method  is  supposed  to 
accomplish  but  does  it  more  effectively, 

•  The  basic  principles  of  the  method  are  sound  and  the  method  itself  well  motivated,  but  in 
use,  practitioners  do  not  apply  the  principles  properly  and  the  method  does  not  live  up  to 
its  potential. 

For  Lind’s  critique  to  have  any  value,  it  must  establish  that  Abstraction-Decomposition  analysis 
fails  on  at  least  one  of  these  criteria.  I  have  concluded  from  my  review  of  his  papers  that  he 
believes  the  method  is  well  motivated  but  that  its  basic  principles  are  fatally  flawed.  In  what 
follows,  I  will  first  assess  the  merit  of  his  argument  in  relation  to  that  fatal-flaws  criterion. 

FATAL  FLAWS 

Abstraction-Decomposition  analysis  is,  as  the  term  implies,  an  analytic  method  and  must 
therefore  be  coupled  with  a  design  or  development  strategy  to  achieve  a  pragmatic  result. 
Ecological  Interface  Design  is  the  design  strategy  of  choice.  Vicente  (2002)  has  reviewed  the 
contributions  of  Ecological  Interface  Design  and  has  concluded  that  progress  has  been 
encouraging,  and  that  there  is  evidence  both  of  applicability  to  a  diverse  set  of  operational 
domains  and  of  technology  transfer  to  industry.  Vicente’s  review  shows  explicit  links  between 
Abstraction-Decomposition  analysis  and  Ecological  Interface  Design  for  at  least  some  of  his 
examples.  When  coupled  with  work  outside  the  Ecological  Interface  Design  realm  by  Naikar, 
Pearce,  Drumm,  and  Sanderson  (2003)  and  Naikar  and  Sanderson  (2001)  relating  to  design  of 
complex  operational  systems,  the  support  for  Abstraction-Decomposition  analysis  is  persuasive. 

Any  argument  for  fatal  flaws  would  have  to  demonstrate  how  these  projects  achieved  successful 
outcomes  in  spite  of — rather  than  because  of — their  reliance  on  Abstraction-Decomposition 
analysis.  Lind  (2003)  did  not  examine  any  of  the  projects  reviewed  by  Vicente,  nor  did  he  assess 
the  work  of  Naikar  and  Sanderson  (2001).  Doubtless,  the  work  of  Naikar  et  al.  (2003)  was 
published  too  late  to  for  him  to  evaluate  in  his  paper,  but  it  also  undermines  his  critique.  In  the 
absence  of  any  substantive  argument  that  can  discount  Vicente’s  conclusions  or  the  relevance  of 
the  work  by  Naikar  and  Sanderson  (2001)  and  Naikar  et  al.  (2003, 1  discount  the  fatal-flaws 
argument. 

Nevertheless,  it  would  be  useful  to  examine  the  content  of  Lind’s  arguments  in  light  of  a  less 
stringent  criterion  noted  above;  that  the  method  does  not  live  up  to  its  potential  because 
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application  of  its  principles  is  inconsistent.  That  exercise  may  serve  to  draw  some  value  from 
Lind’s  critique  by  making  the  principles  more  explicit.  I  am  unaware  of  any  alternative  method 
for  mapping  workplace  structure  and  therefore  do  not  examine  the  possibility  that  there  is  a  more 
effective  alternative. 


TERMINOLOGY 

I  like  to  refer  to  the  product  of  an  Abstraction-Decomposition  analysis  as  an  Abstraction- 
Decomposition  map,  an  Abstraction-Decomposition  space  or  an  Abstraction-Decomposition 
representation.  Elsewhere,  it  is  known  as  an  Abstraction  Hierarchy  or  an  Abstraction- 
Decomposition  model.  The  term  Abstraction  Hierarchy  is  unsatisfactory  because  it  encourages 
neglect  of  the  decomposition  dimension,  which  is  essential  to  this  analysis.  I  dislike 
characterizing  this  as  a  model  because  to  many  the  word  model  implies  properties  that  the  result 
of  this  analysis  does  not  capture,  for  example  properties  of  causality  and  activity.  That  is  not  to 
argue  that  model  is  incorrect  when  used  in  this  sense,  but  only  that  it  introduces  avoidable 
ambiguity. 

Lind  (2003)  refers  to  means-ends  and  part- whole  abstractions.  Abstract  means  to  consider  apart 
from  concrete  existence  (Houghton  Mifflin,  2000).  There  are  numerous  ways  of  abstracting  a 
domain,  and  the  means-ends  relationship  defines  one  of  them.  Neither  assembly  from  parts 
(composition)  nor  disassembly  into  parts  (decomposition)  constitutes  a  gradation  from  concrete 
existence,  and  should  not  be  characterized  as  an  abstraction.  To  speak  of  a  part-whole 
abstraction  suggests  a  failure  to  grasp  the  essential  nature  of  abstraction. 

The  belief  that  cognitive  activity  maps  to  an  Abstraction-Decomposition  structure  in  a  specific 
and  significant  way  is  one  of  the  foundations  for  Cognitive  Work  Analysis.  Both  Rasmussen  and 
Vicente  argue  that  experts  navigate  through  an  Abstraction-Decomposition  space  as  they 
troubleshoot  or  solve  problems.  I  believe  that  much  of  the  confusion  and  skepticism  about  the 
Abstraction-Decomposition  analysis  emanates  from  a  failure  in  the  community  that  practices 
Cognitive  Work  Analysis  to  develop  and  stress  this  idea.  I  suggest  that  if  we  were  to  develop 
and  establish  this  idea  beyond  the  cursory  treatment  given  it  by  both  Rasmussen  and  Vicente,  the 
sense  of  the  Abstraction-Decomposition  analysis  would  become  more  widely  apparent.  I  do  not, 
however,  focus  on  that  issue  in  this  note. 

I  have  long  suspected  that  some  of  the  skepticism  I  encounter  regarding  the  Abstraction- 
Decomposition  analysis  results  from  confusion  about  what  is  meant  by  hierarchy  and  network 
and  I  take  the  opportunity  here  to  clarify  those  terms. 

A  hierarchy  is  a  system  of  ranking  and  organizing  things  in  terms  of  a  relationship,  such  as  is 
superior  to,  is  part  of,  or  is  taller  than.  A  node  at  a  higher  level  of  a  hierarchy  is  designated  as 
superior  to  nodes  to  which  it  is  linked  at  a  lower  level,  and  those  lower-level  nodes  are 
designated  as  subordinate.  A  hierarchy  is: 

•  transitive — if  a  is  superior  to  b,  and  b  is  superior  to  c,  then  a  is  superior  to  c 

•  irreflexive — no  entry  in  the  hierarchy  is  superior  to  itself 

•  asymmetric — if  a  is  superior  to  b,  then  b  is  not  superior  to  a 
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Most  hierarchies  conform  to  the  property  of  containment  in  which  subordinate  nodes  are  strictly 
nested  within  superior  nodes  (Figure  G-l,  left  panel)  but  a  functional  abstraction  hierarchy  does 
not  conform  to  this  property;  subordinate  nodes  need  not  be  contained  by  (linked  to)  only  one 
superior  node  (Figure  G-l,  right  panel).  Relaxation  of  the  containment  property  allows  us  to 
track  multiple  (sometimes  unintended  and  undesirable)  effects  of  subordinate  nodes  and  is 
crucial  to  effective  use  of  means-ends  relationships  for  design  of  socio-technical  systems.  For 
this  reason,  we  should  speak  of  means-ends  rather  than  means-end  relations. 


Figure  G-l.  Hierarchies  generally  conform  to  the  property  of  containment  (left)  but  a 
functional  abstraction  hierarchy  does  not  (right). 

The  classification  hierarchy  of  Figure  G-l  (left  panel)  is  also  an  abstraction  hierarchy  of  the  type 
used  as  a  foundation  for  Ayn  Rand’s  Objectivist  Epistemology  (Rand,  1979/1990).  The 
Abstraction-Decomposition  analysis  of  Cognitive  Work  Analysis  is  about  functional  abstraction 
and  functional  decomposition.  In  discussion,  the  term  functional  is  often  left  implicit  to  avoid 
the  repetition  of  a  long  and  clumsy  designation,  but  it  should  not  be  forgotten. 

A  network  is  an  interconnected  arrangement  of  elements.  The  nodes  may  be  connected  in  either 
a  regular  or  an  irregular  pattern  (e.g.,  a  network  of  railroads,  an  espionage  network,  an  extended 
group  of  people  with  similar  interests  or  concerns  who  interact  and  remain  in  contact  for  mutual 
assistance  or  support).  The  definition  of  network  offered  by  Houghton  Mifflin  (2000)  implies 
that  network  nodes  are  specified  at  a  single  level  of  hierarchy.  The  term,  Functional  Abstraction 
Network  (Elm,  Roth,  Potter,  Gualtieri  &  Easter,  2005)  appears  to  distort  the  concept  of  network. 

COMMENTARY  ON  LIND’S  CRITIQUE 

Lind  (2003)  identified  several  issues,  some  of  which  he  characterized  as  methodological  and 
others  as  conceptual.  Following  the  sequence  used  by  Lind  (2003),  I  will  address  the 
methodological  issues  before  the  conceptual  issues. 

Methodological  Issues 

In  this  section,  T  paraphrase  the  more  significant  of  Lind’s  methodological  issues  as  stated  in  his 
2003  paper  and  offer  my  commentary.  Note  that  my  statement  of  the  issues  is  not  a  quote  but 
rather  my  summary  of  Lind’s  concern.  My  response  to  that  concern  is  formatted  as  a  bullet 
point. 
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Issue:  There  are  no  procedures  or  guidelines  for  knowledge  acquisition. 

•  In  a  design  effort,  we  need  to  acquire  knowledge  and  then  represent  or  summarize  it  in  a 
form  that  can  support  design.  As  noted  by  Bums  and  Vicente  (2001),  the  primary  thrust 
of  expositions  of  Cognitive  Work  Analysis  has  been  on  representation.  This  could  be 
seen  as  neglectful  but  Cognitive  Work  Analysis  is  part  of  the  larger  enterprise  of 
Cognitive  Systems  Engineering,  which  has  a  plethora  of  Knowledge  Acquisition 
methods.  Those  who  practice  Cognitive  Work  Analysis  select  from  those  methods  and, 
given  the  extensive  treatment  of  those  methods  elsewhere,  there  seems  little  need  to 
elaborate  on  them  in  expositions  of  Cognitive  Work  Analysis. 

Issue:  There  is  no  process  for  building,  revising,  modifying  and  validating  models. 

•  Processes  for  building,  revising,  modifying  and  validating  models  are  always  incomplete 
but  Vicente  has  offered  many  details  for  Cognitive  Work  Analysis.  His  guidelines  for 
constructing  an  Abstraction-Decomposition  map  are  detailed  (Vicente,  1999,  pp  165-6). 
The  processes  and  guidelines  offered  by  Lind  (1994,  1999b)  for  Multilevel  Flow 
Modeling  are  no  more  explicit  or  extensive.  In  addition,  some  in  our  community 
continue  to  develop  and  extend  guidelines  for  different  stages  of  Cognitive  Work 
Analysis  (e.g.,  Naikar,  Hopcroft  &  Moylan,  2005). 

Issue:  There  are  no  convincing  arguments  for  the  number  of  means-end  abstraction  levels  or 
part- whole  levels.  It  is  a  strange  coincidence  that  the  number  of  levels  along  the  two  dimensions 
is  the  same. 

•  Pragmatically  speaking,  there  are  five  levels  of  abstraction.  The  limits  are  anchored  by 
the  Why-What-How  sequence.  Purpose  is  the  ultimate  end  and  so  represents  the  upper 
limit.  Physical  material  represents  the  lower  limit.  Objects,  functions,  values  and 
purposes  are  conceptually  different  and  we  further  find  it  useful  to  distinguish  physical 
functions  from  purpose-related  functions.  These  distinctions  should  not  be  considered 
inviolate  because  identification  of  more  appropriate  distinctions  is  always  possible,  but 
they  do  seem  to  correspond  to  the  way  experts  conceptualize  their  work.  Note  that  this  is 
a  pragmatic  issue  (the  distinctions  correspond  to  how  experts  think )  rather  than  a 
metaphysical  one  (the  distinctions  do  not  reflect  an  inherent  structure  of  the  world). 

•  Except  in  Lind’s  own  papers,  I  have  never  seen  a  claim  of  five  decomposition  levels  and 
it  is  definitely  not  a  principle  of  the  Abstraction-Decomposition  analysis.  Levels  of 
decomposition  are  selected  based  on  the  knowledge  acquisition  protocols.  Analysis 
extends  to  a  level  found  useful  for  domain  experts. 

The  inclusion  of  control  systems  in  the  Abstraction-Decomposition  map  is  a  controversial  issue. 

•  Lind  attributes  the  controversy  to  incompatible  statements  made  by  Vicente,  Rasmussen, 
Sanderson  and  Miller.  He  takes  Miller  and  Sanderson  (2000)  to  task  because  they,  in 
forwarding  a  claim  that  the  Abstraction-Decomposition  analysis  cannot  cope  with 
biological  systems,  imply  that  process  plants  do  not  incorporate  control  systems.  Miller 
(personal  communication)  has  indicated  that  she  and  Sanderson  had  not  meant  to  imply 
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that,  and  she  now  believes  that  the  term  entangled  is  a  better  descriptor  for  the  biological 
control  systems  problem. 

•  Lind  takes  Vicente  (1999,  p  9)  to  task  because  of  Vicente’s  definition  of  a  Work  Domain 
as  a  system  being  controlled,  independent  of  any  particular  worker,  automation,  event, 
task,  goal,  or  interface.  Lind  takes  this  definition  to  mean  that  control  systems  should  be 
excluded  from  an  Abstraction-Decomposition  analysis,  but  I  take  this  definition  more 
generally  to  mean  that  agency  should  be  excluded  from  the  analysis.  Thus,  control 
systems  are  not  to  be  analyzed  as  causal  loops  within  the  Abstraction-Decomposition 
analysis.  Control  systems  realize  a  function  and  that  function — together  with  the 
appropriate  decomposition — should  be  included  in  the  Abstraction-Decomposition  map, 
but  the  causal  loop  must  be  investigated  through  some  other  form  of  analysis. 

•  From  my  reading  of  Lind’s  papers,  I  understand  that  analysis  of  processes  within  a 
control  system  is  the  role  he  has  set  for  Multilevel  Flow  Modeling.  If  that  is  the  case, 
Multilevel  Flow  Modeling  and  Abstraction-Decomposition  analysis  do  not  compete  for 
the  same  ground  and  my  email  exchanges  with  Lind  suggest  to  me  that  he  would  agree. 
Bums  and  Vicente  (2001)  also  argue  that  these  two  analyses  yield  different  information. 

Conceptual  problems 

Much  of  what  concerns  Lind  in  this  section  of  his  critique  relates  to  semantics.  In  the  first 
paragraph  of  this  section  he  states,  the  repository  of  concepts  used  to  characterize  the  content  of 
the  five  means-end  levels  is  a  major  source  of  confusion  (Lind,  2003,  p  73).  This,  indeed,  is  the 
single  point  he  makes  that  I  find  telling.  The  confusion  he  expresses  about  the  semantics  that 
underpin  Abstraction-Decomposition  analysis  is  understandable.  The  Cognitive  Engineers  who 
practice  Abstraction-Decomposition  analysis  are,  unfortunately,  inconsistent  in  their  use  of 
words.  Vicente  (1999)  has  made  a  systematic  and  disciplined  attempt  to  clarify  the  semantics 
and  his  book  remains  the  benchmark  for  defining  relevant  concepts. 

As  one  might  imagine,  others  do  not  always  follow  Vicente  precisely.  In  itself,  this  is  not 
problematic  because  we  should  expect  that  usage  of  concepts  would  evolve  as  we  develop  this 
technique,  but  many  analysts  depart  from  Vicente's  terminology  for  no  apparent  reason,  without 
explanation,  and  without  acknowledging  the  departure.  I  am  left  with  the  impression  that  there  is 
a  troubling  lack  of  discipline  in  our  community  regarding  the  meaning  of  terms  and  that 
relatively  few  are  concerned  by  that  state  of  affairs.  For  example,  in  response  to  my  expression 
of  concern  regarding  our  casual  use  of  words  (Lintem,  2004),  John  Flach  has  argued  that  it  is  an 
issue  of  which  is  to  be  the  master  (Flach,  2004),  presumably  implying  that  we  can  use  words  in 
any  way  we  desire. 

I  find  this  attitude  as  troubling  as  Lind  (2003)  apparently  does.  He  notes  our  use  of  the  term 
function  and  argues  that  we  do  not  recognize  its  multiple  meanings.  The  same  can  be  said  of  the 
term  process  and  it  is  a  further  concern  that  there  is  overlap  in  some  of  these  meanings  between 
the  two  terms.  Nevertheless,  Vicente  (1999)  defines  function  in  the  manner  in  which  he  intends 
it  to  be  used  in  Work  Domain  Analysis,  and — while  he  does  not  specifically  define  process — his 
definition  of  product  model  indicates  what  he  means  by  process  (Box  1).  There  is  also  confusion 
about  the  distinction  between  purpose  and  goal,  but  again  Vicente  defines  the  manner  in  which 
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they  can  be  distinguished  (Box  2).  Nevertheless,  there  are  many  examples  in  the  literature  of 
Abstraction-Decomposition  analysis  completed  since  the  publication  of  Vicente's  book  in  which 
process  is  equated  to  function  and  goal  is  substituted  for  purpose. 


Box  1:  Function  &  Process 

Function  -  a  goal-relevant  structural  property  of  a  Work  Domain.  An  Affordance 
that  is  relevant  to  the  Purposes  for  which  the  Work  Domain  was  designed 
(Vicente,  1999,  p  6). 

Product  Model  -  a  black-box  Model  describing  the  Behavior  of  a  System  but  not 
the  process  or  mechanism  by  which  that  Behavior  is  generated  (i.e.,  "what",  but 
not  "how").  A  Model  of  System  Behavior  rather  than  System  Structure.  (Vicente, 
1999,  p  7). 

Comment: 

By  this  definition  of function,  it  is  a  structural  property  whereas  the  usage  of 
process  in  this  definition  of  Product  Model  treats  it  as  an  action  property 


Box  2:  Purpose  &  Goal 

Purposes  -  the  overarching  intentions  that  a  Work  Domain  was  designed  to 
achieve.  Note  that  Purposes  are  properties  of  Work  Domains,  not  Actors,  and  that 
they  are  relatively  permanent  (unlike  the  Goals  of  Actors,  which  change  over  time) 
(Vicente,  1999,  p  8). 

Goal  -  a  State  to  be  achieved,  or  maintained,  by  an  Actor  at  a  particular  time.  Note 
that  goals  are  attributes  of  Actors,  not  Work  Domains,  and  that  they  are  dynamic 
(unlike  the  Purposes  for  which  a  Work  Domain  is  built,  which  are  relatively 
permanent)  (Vicente,  1999,  p  6). 


This  lack  of  discipline  in  use  of  words  is  particularly  troublesome  for  the  practice  of  Abstraction- 
Decomposition  analysis  because  this  method  generates  so  much  controversy.  Our  continuing 
lack  of  discipline  in  this  area  can  only  serve  to  confuse  those  we  are  trying  to  inform  and  leave 
us  open  to  the  sort  of  criticism  that  Lind  has  leveled. 

Many  of  the  conceptual  issues  Lind  identifies  do  not  emanate  from  unclear  and  inconsistent  use 
of  terminology  and  I  respond  to  those  issues  below  in  the  same  manner  I  responded  to  the 
methodological  issues. 

Issue:  A  means-ends  relation  has  causal  properties  but  the  Abstraction-Decomposition  analysis 
does  not  deal  with  causes. 

•  Vicente  (1999,  p  7)  is  unambiguous.  He  refers  to  the  structural  means  available  for 
achieving  the  ends  (Box  3).  This  is  consistent  with  the  common  language  interpretation 
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of  the  means  test  (Houghton  Mifflin,  2000),  which  essentially  asks  whether  you  have  the 
resources  that  will  permit  you  to  live  without  additional  resources.  Vicente’s  treatment  of 
means-ends  excludes  any  consideration  of  causality,  which  is  not  to  claim  causality  is 
irrelevant  but  to  claim  that  its  analysis  is  undertaken  elsewhere. 


Box  3:  Means-Ends  Relation 

Means-Ends  Relation;  the  relationship  between  adjacent  levels  in  a  Means-Ends 
Hierarchy.  The  level  below  a  given  level  describes  the  structural  means  that  are 
available  for  achieving  the  level  above.  The  level  above  a  given  level  describes  the 
ends  (or  Functions)  that  can  be  achieved  by  the  level  below  (Vicente,  1999,  p  7). 


Issue:  The  combination  of  means-end(s)  and  causality  concepts  is  inconsistent  with  the  intrinsic 
logic  of  many-to-one  mappings. 

•  A  key  benefit  of  the  Abstraction-Decomposition  map  is  that  it  reveals  complex  mappings; 
many-to-one,  many-to-many  and  one-to-many.  The  standard  Systems  Engineering 
strategy  of  assigning  Integrated  Product  Teams  to  different  functional  areas  prevents 
mapping  of  subtle  and  unexpected  interdependencies  between  functional  areas.  To  my 
knowledge,  the  Abstraction-Decomposition  map  is  the  only  representation  available 
today  that  can  reveal  these  interdependencies  and  it  does  so  by  virtue  of  allowing 
complex  mappings.  As  noted  above,  means-ends  relations  are  not  causal.  The 
incompatibility  of  causal  concepts  with  complicated  mappings  is  one  reason  that 
practitioners  of  Cognitive  Work  Analysis  do  not  enter  causal  concepts  into  their 
Abstraction-Decomposition  maps. 

Issue:  It  is  important  to  distinguish  between  different  types  of  means-ends  relations. 

•  Again,  Vicente  (1999,  p  7)  is  unambiguous.  There  is  one  type  of  means-ends  relation. 
Would  others  be  useful  and  could  they  be  incorporated  into  the  Abstraction- 
Decomposition  analysis?  Resolution  of  that  question  would  require  extensive  exploration 
but  I  doubt  it  would  be  a  productive  exercise.  Lind  (1999b)  offers  a  number  of  means- 
ends  relations  for  Multilevel  Flow  Modeling  and  they  are  possibly  useful  for  the  form  of 
technical  analysis  he  undertakes,  but  the  distinctions  he  makes  have  no  obvious 
implications  for  the  design  of  Human-Systems  Interaction. 

Issue:  The  semantics  of  the  means-ends  and  causal  relations  in  the  Abstraction-Decomposition 
map  allows  circular  plant  descriptions. 

•  The  issue  of  circular  description  is  one  of  the  reasons  given  by  Elm  (2002)  for  the 
Functional  Abstraction  Network  developed  in  Applied  Cognitive  Work  Analysis  as  an 
alternative  to  the  Abstraction-Decomposition  map.  It  is  possibly  no  accident  that  the 
Functional  Abstraction  Network  developed  by  Elm  et  al  (2005)  has  some  of  the 
characteristics  of  a  Multilevel  Flow  Model,  including  references  to  causality  (Elm,  2002). 
However,  circular  descriptions  are  not  valid  in  an  Abstraction-Decomposition  map  and 
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those  who  note  it  as  a  problem  do  so  because  they  do  not  understand  the  nature  of  means- 
ends  relations  as  defined  by  Vicente — and  do  not  recognize  the  significance  of  the 
complex  mappings.  Neither  Multilevel  Flow  Modeling  nor  the  Functional  Abstraction 
Network  of  Applied  Cognitive  Work  Analysis  can  depict  functional  interdependencies. 
That  does  not  invalidate  their  use  as  tools  for  design  of  Human-Systems  Integration  but 
those  tools  do  not  substitute  for  an  Abstraction-Decomposition  map. 

Issue:  The  inclusion  of  actions  on  the  level  of  physical  function  in  the  Abstraction  Hierarchy 
(Rasmussen  et.  al.,  1994)  is  problematic.  Most  people  would  regard  actions  as  genuine  means 
(consider  e.g.  the  following  sentence  “the  turning  of  the  valve  by  30  degrees  is  a  means  to 
increase  the  flow  of  water’'’)  but  actions  does  not  to  fit  naturally  in  the  same  category  as  material 
objects  like  pumps  and  valves. 

•  I  remain  uncertain  whether  Lind  meant  to  attribute  that  quote  to  Rasmussen  et  al.,  (1994) 
but  I  could  not  find  it.  Vicente  (1999)  is  clear  on  the  fact  that  action  statements  are  not  to 
be  included  in  the  Abstraction-Decomposition  map  and  I — and  at  least  some  others — 
follow  this  guidance  rigorously.  In  my  view,  Vicente’s  recommendation  is  consistent 
with  the  exposition  of  Rasmussen  et  al.,  (1994).  It  is  an  unfortunate  characteristic  of  the 
English  language  that  certain  words  can  signify  either  functions  or  actions  (e.g.  landing 
as  relevant  to  aircraft)  and  I  sometimes  notice  words  that  have  this  characteristic  in 
Rasmussen’s  and  Vicente’s  expositions.  I  have  not  found  Rasmussen  to  be  as 
unambiguously  explicit  as  Vicente  but  I  do  not  find  noteworthy  conceptual 
incompatibilities  in  their  respective  treatments  of  Abstraction-Decomposition  analysis. 
Lind’s  claim  that  actions  do  not  to  fit  naturally  in  the  same  category  as  material  objects  is 
consistent  with  Vicente’s  position  and,  I  believe,  with  Rasmussen’s. 

•  The  search  for  conceptual  incompatibilities  between  Rasmussen  and  Vicente  is 
nevertheless,  an  unfortunate  exercise.  We  should  hope  that  we  are  developing  the  tools 
of  Cognitive  Work  Analysis  and  I  doubt  that  anyone,  including  either  Rasmussen  or 
Vicente,  would  imagine  that  any  of  the  earlier  treatments  are  flawless.  From  that 
perspective,  we  would  hope  that  the  more  recent  expositions  refine  issues  and  correct 
inconsistencies. 

Lind  takes  the  view  that  confusion  about  semantics  results  from  attempts  to  generalize  to  a 
number  of  work  domains  but  I  continue  to  believe  that  the  potential  to  generalize  across  work 
domains  is  a  major  strength  of  the  Abstraction-Decomposition  analysis.  Lind  also  takes  the  view 
that  clarification  of  the  semantics  will  restrict  the  range  of  domains  to  which  the  Abstraction- 
Decomposition  analysis  is  applicable,  while  I  take  the  view  that  clarification  of  the  semantics 
will  extend  the  range  and  value  of  application.  These  are  unsupported  claims  but  we  should  note 
that  no  research  endeavor  could  progress  without  a  number  of  strategic  commitments  of  faith. 

Some  of  Lind’s  critique  is  premature,  but  at  some  stage  it  is  essential  that  the  Abstraction- 
Decomposition  analysis  be  shown  to  contribute  to  the  design  of  cognitive  systems.  Some  strong 
examples  of  success  are  already  available  and  have  been  noted  earlier  in  this  paper.  Work  is 
ongoing  and  as  that  is  reported  in  the  public  domain,  we  will  be  able  to  update  our  ideas  about 
this  form  of  analysis.  However,  I  remain  unaware  of  any  competing  analysis  that  is  devoted 
exclusively  to  mapping  out  functional  structure,  and  part  of  the  disagreement  may  be  about 
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whether  it  is  useful  to  map  the  functional  structure.  That  may  be  a  focus  for  future  discussion; 
suffice  to  say  at  this  stage  that  those  who  undertake  Cognitive  Work  Analysis  believe  it 
important. 

This  problem  of  semantic  clarification  is  an  issue  that  we,  as  a  community  of  practitioners,  have 
not  taken  seriously  and  we  could  do  well  to  use  Lind’s  critique  of  conceptual  problems  as  one 
guide  in  formulating  an  agenda.  As  I  have  noted  however,  there  are  several  things  that  Lind  says 
about  semantics  that  would  lead  us  in  the  wrong  direction.  Furthermore,  he  offers  a  number  of 
remarks  in  the  Conceptual  Problems  section  of  his  paper  that  reveal  a  distorted  understanding  of 
the  role  and  application  of  the  Abstraction-Decomposition  analysis.  We  need  to  sort  through 
these  remarks  to  identify  those  that  make  sense  if  we  are  to  be  guided  rather  than  distracted  by 
them. 


MULTILEVEL  FLOW  MODELING 

...the  interaction  between  automated  controls  and  their  responses  to  operator  intervention  can 
only  be  understood fully  if  it  is  seen  as  a  goal  oriented  activity.  Lind,  1 999,  page  171 

Lind  (1999a;  2003)  offers  Multilevel  Flow  Modeling  as  a  better  strategy  for  dealing  with 
functional  abstraction,  functional  decomposition,  and  means-ends  relations.  The  main  objectives 
of  Multilevel  Flow  Modeling  are  to  develop  concepts  and  methods  for  modeling  of  complex 
industrial  artifacts  and  to  use  the  models  in  conceptual  design  of  industrial  automation  systems, 
including  intelligent  controls  and  supervisory  functions  for  the  operator  (Lind,  2003,  p  67).  The 
strategy  is  to  represent  goal  structures  and  their  relationship  to  underlying  causal  mechanisms 
of  the  plant  in  a  formalized  way  (Lind  1999b). 

A  tutorial  example  from  Lind  (1994)  illustrates  some  of  the  basic  concepts  and  the  strategy. 
Figure  G-2  depicts  a  system  with  two  interacting  feedback  loops.  Cold  and  hot  inflows  of  fluid 
are  mixed  to  produce  an  outflow  at  a  preset  temperature  and  flow  rate.  Feedback  loops  through 
Sl-Rl  to  control  valve  VI  and  S2-R2  to  control  valve  V2  ensure  that  the  system  settles  on  a 
stable  flow  at  the  preset  temperature. 

The  Multilevel  Flow  Model  of  Figure  G-3  formally  depicts  the  mass  and  energy  flow  structures 
involved  in  mixing  of  the  two  streams  of  water  and  the  two  feedback  control  functions  involved 
in  balancing  the  temperature  and  flow  rate.  Connections  show  how  system  functions  are 
integrated  to  satisfy  system  goals.  Connections  identify  means-ends  relations,  but  in  contrast  to 
the  Abstraction-Decomposition  map  as  described  by  Vicente  (1999),  there  are  eight  types  of 
means-ends  relations.  One  corresponds  to  the  type  used  in  the  Abstraction-Decomposition  map 
and  another  corresponds  to  a  causal  link. 
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Figure  G-2.  A  two-variable  process  with  two  feedback  loops  (Lind,  1994) 


Figure  G-3.  A  Multilevel  Flow  Model  of  the  two-variable  process,  two-feedback  loop 

system 
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The  examples  offered  by  Lind  (1994,  1999b)  are  primarily  technical  and  offer  little  in  the  way  of 
socio-technical  analysis.  Unlike  the  Abstraction-Decomposition  map,  which  is  confined  to  a 
representation  of  structure,  a  Multilevel  Flow  Model  represents  both  process  and  structure. 

Bums  and  Vicente  (2001),  in  their  contrast  of  Abstraction-Decomposition  analysis  to  Multilevel 
Flow  Modeling,  concluded  that  the  former  produces  a  work  domain  structure  model  and  the 
latter  a  work  domain  goal  model.  They  note  that  different  kinds  of  analyses  yield  different 
information.  Thus,  the  choice  of  method  should  be  determined  in  part  by  what  sort  of 
information  is  needed. 


Burns  and  Vicente  (2001) 

The  paper  by  Bums  and  Vicente  (2001)  offers  another  tutorial  example  on  Multilevel  Flow 
Modeling  and  also  a  detailed  comparison  of  Multilevel  Flow  Models,  Abstraction- 
Decomposition  maps,  and  Decision  Ladder  representations.  It  is  an  evocative  and  succinct 
paper.  I  recommend  it  be  read  in  full,  but  I  provide  a  brief  explanation  of  the  ideas  in  that  paper 
below. 

The  purpose  of  analysis  is  to  abstract  from  specific  details  to  provide  a  generic  description.  In 
Cognitive  Engineering,  that  description  should  have  implications  for  design.  Vicente  (1999)  has 
argued  that  analysis  can  be  of  tasks  or  work  domains.  Task  descriptions  identify  actions  that  can 
or  should  be  performed  by  one  or  more  agents.  Work  domain  descriptions  come  in  two  forms; 
they  identify  either  structure  or  goals. 

A  Multilevel  Flow  Model  is  of  the  goal  type;  it  reveals  how  structures  are  connected  to  goals 
(desired  states).  The  connections  are  causal  links  that  can  be  characterized  as  goal-achievement 
relations.  The  description  is  not  hierarchic;  it  allows  circular  descriptions  of  the  form  that  would 
be  needed  to  describe  the  information  and  energy  flows  within  a  closed  loop  system  (e.g.  a  home 
heating  system). 

An  Abstraction-Decomposition  map  is  a  work  domain  structure  model;  it  does  not  include  goals 
or  actions.  "The  purpose  or  functions  of  the  work  domain  are  connected  via  structural  means-end 
relations  to  functions  or  objects  at  the  next  level  down  the  hierarchy.  Levels  are  connected  in  a 
Why-What-How  relationship  where  entries  at  the  next  highest  level  show  why  a  function  or 
physical  resource  is  in  the  work  domain,  and  entries  at  the  next  lowest  level  identify  the 
resources  needed  to  realize  a  purpose  or  function  (Figure  G-4). 

In  comparison  to  a  Multilevel  Flow  Model,  the  Abstraction-Decomposition  map  offers  a 
description  of  purpose  rather  than  goals.  The  description  is  hierarchic,  moving  from  abstract  at 
the  top  to  concrete  at  the  bottom.  There  are  no  circular  loops  between  levels  and  each  level 
offers  a  different  kind  of  description.  The  structural  means-ends  relations  are  directional  with 
ends  ordered  above  means  throughout. 

A  Multilevel  Flow  Model  reveals  states  important  to  predefined  modes  of  operation  but  does  not 
reveal  the  purpose  of  the  work  domain.  A  Multilevel  Flow  Model  indicates  the  roles  that  certain 
pieces  of  equipment  play  in  achieving  operational  states.  That  could  aid  development  of  an 
equipment  use  procedure,  but  it  would  be  a  procedure  that  would  be  brittle  in  an  unanticipated 
situation  because  it  is  based  on  event-dependent  goals. 
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Figure  G-4.  An  abstraction  hierarchy  for  a  portable  music  system  (IPOD)  showing  the 
Why-What-How  links  for  the  means  end  relations. 


The  Abstraction-Decomposition  analysis  provides  an  event-independent  description;  it  reveals 
how  different  material  resources  and  their  functions  support  system  purpose.  Action  possibilities 
may  be  detected  through  an  understanding  of  how  equipment  works,  but  specific  courses  of 
action,  target  goals,  or  equipment  usage  are  not  specified. 

Cognitive  Work  Analysis  distinguishes  structural  from  task  analyses.  The  task  description  of 
choice  is  the  Decision  Ladder.  As  in  a  Multilevel  Flow  Model,  a  Decision  Ladder  shows  desired 
operational  states  and  activity  required  to  realize  them  but  the  two  forms  of  description  differ  in 
content.  A  Multilevel  Flow  Model  uses  goal-achievement  relations  to  connect  desired 
operational  states,  while  a  Decision  Ladder  uses  information  links  to  depict  a  task  as  a  sequence 
of  desired  operational  states.  The  Decision  Ladder  offers  a  richer  description  of  cognitive  and 
action  states  by  outlining  information  processing  stages  and  cognitive  shortcuts.  Unlike  an 
Abstraction-Decomposition  map,  the  Decision  Ladder  has  loops  and  cycles  and  is  not  hierarchic. 

Functional  Abstraction  Network 

I  remain  unclear  regarding  the  strength  of  the  connection  between  Applied  Cognitive  Work 
Analysis  and  Multilevel  Flow  Modeling,  but  similarities  are  evident.  In  particular,  the 


96 


Functional  Abstraction  Network  of  Applied  Cognitive  Work  Analysis  bears  a  striking 
resemblance  to  a  Multilevel  Flow  Model.  Within  the  community  of  Applied  Cognitive  Work 
Analysis  practice,  the  intellectual  debt  to  both  Rasmussen  and  Lind  is  acknowledged.  However, 
presumably  in  reference  to  Abstraction-Decomposition,  it  is  said  that  the  term  “hierarchy”  is 
actually  a  misnomer — the  structure  of  the  model  is  actually  a  NETWORK  (Elm,  2002).  This 
claim  is  both  distracting  and  misleading.  The  product  of  Applied  Cognitive  Work  Analysis  is  a 
network  but  the  product  of  Abstraction-Decomposition  analysis  is  a  hierarchy,  albeit  a  two- 
dimensional  hierarchy  in  which  one  of  the  dimensions  (i.e.  abstraction)  does  not  conform  to  the 
common  hierarchical  property  of  containment. 

SUMMARY 
Lind’s  Critique 

I  am  troubled  by  the  tone  of  Lind’s  critique;  it  is  predominantly  negative  and,  while  offering 
some  useful  observations,  does  so  in  a  tone  that  is  unlikely  to  encourage  those  of  us  working  on 
Abstraction-Decomposition  analysis  to  take  him  seriously.  Unfortunately,  the  content  of  Lind’s 
critique  has  more  weaknesses  than  the  does  the  method  he  critiques.  He  argued  as  if  the  issues 
he  was  dealing  with  constituted  fatal  flaws,  but — even  if  valid — most  of  the  issues  he  raised  did 
not  point  towards  fatal  flaws  but  rather  towards  issues  that  could  be  resolved  and,  if  resolved, 
would  strengthen  the  method. 

The  Cognitive  Engineering  community  is  a  relatively  small  group  of  analysts  and  designers  who 
are  tackling  difficult  problems  in  different  ways.  No  one  has  yet  established  an  approach  that  is 
undeniably  effective  with  the  full  range  of  problems  that  we  face.  Indeed,  subtleties  of  the 
methods  we  use  are  difficult  to  grasp  and  one  needs  to  work  extensively  with  any  one  of  them  to 
gain  any  significant  level  of  appreciation  of  its  value.  As  is  true  of  many  analytic  methods,  it  is 
unlikely  that  a  well-founded  critique  of  such  a  complex  method  as  Abstraction-Decomposition 
analysis  could  come  from  someone  who  has  not  worked  extensively  with  it.  While  Lind  has 
offered  a  number  of  useful  observations,  we  should  be  selective  about  which  of  them  we  take 
seriously.  Any  attempt  to  deal  with  those  based  on  misunderstandings  would  create  more 
confusion  without  adding  value. 

I  recognize  that  Lind  was  involved  in  early  developments  of  Abstraction-Decomposition 
analysis,  but  his  misconceptions  especially  about  the  purpose  and  nature  of  abstraction  and 
means-ends  relations  are  significant.  I  wonder  at  this  but  speculate  that  he  has  not  kept  abreast 
of  developments  over  the  past  15  years  or  so.  In  my  own  view,  the  early  work  as  reported  by 
Rasmussen  (1986)  on  what  might  be  termed  the  Riso  analysis12,  was  fragmented  and  partially 
inconsistent  with  the  later  developments  reported  in  Rasmussen  et  al  (1994)  and  Vicente  (1999). 
The  1986  book  is  radical,  evocative,  and  sometimes  inspirational,  but  the  ideas  it  contains  have 
been  refined  considerably  since  its  publication.  I  suspect  that  Lind  has  not  maintained 
familiarity  with  these  developments. 


12  After  the  Rise  National  Laboratory  at  which  Rasmussen  served  as  a  professor  of  Cognitive  Systems  Engineering. 
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It  should  also  be  noted  that  Rasmussen  typically  takes  a  global  perspective,  emphasizing 
cognitive  systems  (e.g.,  Rasmussen,  et  ah,  1994)  while  Lind  focuses  on  automated  control 
systems.  On  several  occasions,  while  reviewing  Lind’s  papers,  I  puzzled  over  the  origin  of 
certain  statements  and  have  come  to  believe  that  they  emerge  from  Lind’s  techno-centric  view 
and  his  focus  on  automated  control  systems.  That,  however,  does  not  make  him  wrong;  the 
substantive  content  of  his  claims  need  to  be  assessed  and  I  have  sought  to  do  that  in  this 
commentary. 


Abstraction-Decomposition  Analysis 

Abstraction-Decomposition  analysis  has  a  unique  role  to  play.  To  my  knowledge,  no  other 
analytic  method  lays  out  functional  structure  in  a  manner  that  supports  formative  design  of 
complex  socio-technical  systems.  I  have  yet  to  find  anything  comparable  in  Systems 
Engineering.  Only  Gibson  (1979)  and  Rand  (1979/1990)  promote  a  similar  view  and  they  do  not 
propose  any  explicit  means  of  representing  the  spaces  they  conceptualize. 

The  practice  of  Cognitive  Work  Analysis  is  occasionally  criticized  for  its  emphasis  on  Work 
Domain  Analysis — that  often  being  the  only  phase  of  Cognitive  Work  Analysis  that  is 
completed.  In  their  recent  book,  Bums  and  Hajdukiewicz  (2004)  ignore  the  remaining  four 
phases  of  Cognitive  Work  Analysis  on  their  way  to  developing  an  approach  to  Ecological 
Interface  Design.  While  this  emphasis  might  be  seen  as  a  weakness,  it  has  resulted  in  the  relative 
maturation  of  Work  Domain  Analysis  and  its  analytic  product,  the  Abstraction-Decomposition 
map. 

Abstraction  is  a  challenging  concept.  The  potential  contribution  of  the  abstraction  dimension 
may  be  little  appreciated  because  it  is  poorly  understood  and  is  readily  confused  with 
decomposition.  Sarcedoti  (1974)  is  one  who,  in  proposing  a  hierarchy  of  abstraction  spaces  as  a 
means  of  reducing  combinatorial  complexity  for  planning,  appears  at  first  to  appreciate  the 
contribution  of  an  abstraction  analysis.  However,  having  outlined  this  proposition,  Sarcedoti 
then  proceeds  to  treat  abstraction  in  terms  of  decomposition.  I  suggest  that  the  distinction 
between  abstraction  and  decomposition,  although  logical,  is  not  obvious  and  those  who  analyze 
complex  socio-technical  systems  will  not  turn  their  attention  to  it  unless  they  are  encouraged  to 
do  so.  This  is  possibly  a  formative  contribution  that  those  who  are  developing  the  Abstraction- 
Decomposition  analysis  can  make  to  Cognitive  Engineering  practice. 

A  Personal  View  of  Rasmussen’s  Contribution 

Rasmussen’s  most  important  foundational  assumption  relates  to  the  primacy  of  structural 
analysis  of  the  workspace.  In  that  regard  his  is  an  ecological  approach  (Gibson,  1979)  in  which 
structural  analysis  of  the  environment  is  central.  Most  cognitive  analysis  deals  with  processes, 
tasks,  or  activities.  Rasmussen's  approach  does  not  deny  the  value  of  process,  task,  or  activity 
analysis  but  proposes  that  the  structural  analysis  is  essential. 

A  further  important  foundational  assumption  is  that  the  design  of  a  cognitive  workspace  must  be 
structured  to  be  compatible  with  effective  patterns  of  cognitive  work  (e.g.,  problem  solving, 
planning)  and  Rasmussen  proposed  that  a  functional  abstraction-decomposition  space  captures 
the  essential  properties  of  that  workspace.  Those  who  do  not  care  for  the  abstraction- 
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decomposition  structure  must  reject  this  foundational  assumption  of  compatibility  with  the 
pattern  of  human  work,  or  else  propose  alternative  dimensions  for  the  workspace  structure. 

It  should  be  noted  that  the  abstraction-decomposition  space  is  a  construction  abstracted  from 
descriptions  of  cognitive  activity.  It  is  not  a  metaphysical  statement  about  the  nature  of  the 
world  but  rather  a  pragmatic  statement  about  how  we  believe  experts  perceive  their  cognitive 
workspace.  As  with  any  analysis,  it  emphasizes  certain  properties  at  the  expense  of  others.  It 
remains  possible  that  different  dimensions  would  map  the  structure  of  cognitive  work  more 
accurately.  We  might  also  wonder  whether  different  types  of  links  (e.g.,  if-then  relationships, 
Anne  Miller,  personal  communication)  would  capture  reasoning  processes  more  accurately.  Any 
such  speculations  would  have  to  be  evaluated  against  a  suitable  body  of  data  and  it  is  also 
possible  that  the  structure  of  cognitive  work  is  not  generic  across  different  work  domains. 
Progress  of  that  sort  would  not,  however,  invalidate  Rasmussen’s  foundational  assumptions  but 
would  rather  validate  his  structural  assumption  and  his  reliance  for  specific  insights  on 
descriptions  of  expert  cognitive  activity. 


REFERENCES 

Bums,  C.  &  Hajdukiewicz,  J.  R.  (2004).  Ecological  interface  design.  Boca  Raton,  FL:  CRC 
Press. 

Bums,  Catherine  M.  &  Vicente,  Kim  J.  (2001).  Model-Based  Approaches  for  Analyzing 

Cognitive  Work:  A  Comparison  of  Abstraction  Hierarchy,  Multilevel  Flow  Modeling, 
and  Decision  Ladder  Modeling.  International  Journal  of  Cognitive  Ergonomics ,  5(3), 
357-366. 

Elm,  Bill  (2002).  Applied  Cognitive  Work  Analysis:  Applied  Cognitive  Work  Analysis  (A  virtual 
workshop:  What  puts  the  “ applied  ”  in  Applied  Cognitive  Work  Analysis?).  Pittsburgh 
PA:  Cognitive  Systems  Engineering  Center  (CSEC),  ManTech  Aegis  Research 
Corporation. 

Elm,  W.C.,  Potter,  S.S.,  Gualtieri,  J.W.,  Roth,  E.M.  and  Easter,  J.R.  (2003),  Applied  cognitive 
work  analysis:  a  pragmatic  methodology  for  designing  revolutionary  cognitive 
affordances,  in  Handbook  for  Cognitive  Task  Design,  Hollnagel,  E.  (Ed.).  London: 
Lawrence  Erlbaum  Associates. 

Elm,  W.  C.,  Roth,  E.  M.,  Potter,  S.  S.,  Gualtieri,  J.  W.  and  Easter,  J.  R.  (2005).  Applied 

Cognitive  Work  Analysis  (Applied  Cognitive  Work  Analysis).  In  Neville  Stanton,  Alan 
Hedge,  Karel  Brookhuis,  Eduardo  Salas  and  Hal  Hendrick  (Eds.),  Handbook  of  Human 
Factors  and  Ergonomics  Methods,  (pp.  36-1  -  36-9).  Boca  Raton,  FL:  CRC  Press. 

Flach,  John  (2004).  Untitled  review  of  presentations.  Workshop  on  Cognitive  Work  Analysis, 
Seattle,  WA,  November  2004. 

Gibson,  James  J.  (1979).  The  Ecological  Approach  to  Visual  Perception.  Boston:  Houghton 
Mifflin. 


99 


Houghton  Mifflin  (2000).  The  American  Heritage  Dictionary  of  the  English  Language,  4th 
Edition.  Houghton  Mifflin  Company. 

Lind,  Morten  (2003).  Making  sense  of  the  abstraction  hierarchy  in  the  power  plant  domain. 
Cognition,  Technology  and  Work,  5,  67-81. 

Lind,  M.  (1999a).  Making  sense  of  the  abstraction  hierarchy.  Proceedings  of  cognitive  science 
approaches  to  process  control.  In  CSAPC’  99,  Villeneuve  d’Ascq,  France,  21-24 
September  1999. 

Lind,  M.  (1999b).  Plant  modelling  for  human  supervisory  control.  Transactions  of  the  Institute  of 
Measurement  and  Control,  21(No  4/5),  pp  177-180. 

Lind,  M.  (1994).  Modelling  Goals  and  Functions  of  Complex  Industrial  Plant.  Journal  of  Applied 
Artificial  Intelligence,  8,  259-283. 

Lintem,  Gavan  (2004).  Dissensions  in  Work  Domain  Analysis.  Workshop  on  Cognitive  Work 
Analysis,  Seattle,  WA,  November  2004. 

Miller,  A.  &  Sanderson,  P.  (2000).  Modelling  “deranged”  physiological  systems  for  ICU 
information  system  design.  In  Human  Factors  and  Ergonomics  Society  (HFES/1EA 
2000),  San  Diego,  CA.  30  July-4  August  2000. 

Naikar,  Neelam;  Hopcroft,  Robyn  and  Moylan,  Anna  (2005).  Work  Domain  Analysis: 

Theoretical  Concepts  and  Methodology.  (DSTO-TR-1665).  Melbourne,  Victoria, 

Australia:  Aeronautical  &  Maritime  Research  Laboratories,  Defence  Science  & 
Technology  Organisation. 

Naikar,  N.,  Pearce,  B.,  Drumm,  D.  &  Sanderson,  P.  M.  (2003).  Technique  for  designing  teams 
for  first-of-a-kind  complex  systems  with  cognitive  work  analysis:  Case  study.  Human 
Factors,  45(2),  202-217. 

Naikar,  N.  &  Sanderson,  P.  M.  (2001).  Evaluating  designs  proposals  for  complex  systems  with 
work  domain  analysis.  Human  Factors,  43(4),  529-542. 

Rand,  Ayn  (1979/1990).  Introduction  to  Objectivist  Epistemology.  Harry  Binswanger  &  Leonard 
Peikoff  (Eds).  (Expanded  2nd  Edition),  New  York:  New  American  Library. 

Rasmussen,  Jens  (1986).  Information  processing  and  human  machine  interaction:  an  approach 
to  cognitive  engineering.  (North  Holland  Series  in  System  Science  and  Engineering,  12), 
New  York:  Elsevier  Science  Ltd. 

Rasmussen,  J.,  Petjersen,  A.  M.  &  Goodstein,  L.  P.  (1994).  Cognitive  systems  engineering.  New 
York:  John  Wiley. 

Sacerdoti,  E.  D.  (1974).  Planning  in  a  hierarchy  of  abstraction  spaces.  Artificial  Intelligence,  5, 
115-135. 


100 


Vicente,  K.  J.  (2002).  Ecological  Interface  Design:  Progress  and  Challenges.  Human  Factors, 
44(1),  62-78. 

Vicente,  K.  J.  (1999).  Cognitive  Work  Analysis:  Towards  safe,  productive,  and  healthy 
computer-based  work.  Mahwah,  NJ:  Lawrence  Erlbaum  &  Associates. 


101 


APPENDIX  H.  DR.  CUMMINGS’  CRITIQUE  OF  “FORMS  OF  REPRESENTATION 
FOR  COGNITIVE  DEMANDS  IN  SYSTEM  ACQUISITION” 


102 


Background 


Dr.  Mary  (Missy)  Cummings  developed  this  critique  of  “Forms  of  Representation  for  Cognitive 
Demands  in  System  Acquisition”  for  Dr.  Lintem  under  a  subcontract  to  General  Dynamics.  Dr. 
Cummings  is  a  professor  in  the  Massachusetts  Institute  of  Technology’s  Aeronautics  & 
Astronautics  Department.  Her  research  approach  has  been  shaped  by  her  extensive  military 
experience  and  her  experience  as  a  pilot  of  one  of  the  Navy’s  most  advanced  fighter  aircraft.  She 
has  a  strong  interest  in  the  use  of  cognitive  engineering  methods  in  the  design  of  complex 
military  systems. 

The  document  that  she  critiqued,  “Forms  of  Representation  for  Cognitive  Demands  in  System 
Acquisition”  was  actually  the  proposal  that  Dr.  Lintem  submitted  to  AFRL/HECS  for  this  effort. 
Dr.  Cummings  sometimes  refers  to  the  document  as  the  “proposal”  in  this  appendix. 

Executive  Summary 

The  following  report  analyzes  Cognitive  Work  Analysis  (CWA),  with  special  emphasis  on 
command  and  control  decision  support  systems.  Specific  areas  of  investigation  include 
problematic  definitions  and  classifications  associated  with  CWA,  the  incompatibilities  of  CWA 
with  current  systems  engineering  practices  to  include  requirements  generations,  and  limitations 
of  CWA  to  include  1)  embedded  system  representation,  2)  application  to  intentional  domains, 

3)  adaptation  to  revolutionary  systems,  and  4)  ill-defined  phases  of  analysis.  A  section  is 
included  to  discuss  current  time  critical  targeting  issues  and  the  report  concludes  with  a  list  of 
recommendations  for  future  exploration. 

1.0  Definitions 

This  report  begins  with  a  discussion  of  definitions.  While  it  may  seem  a  trivial  and  semantic 
discussion  to  reexamine  definitions  surrounding  Cognitive  Work  Analysis  (CWA)  and  the 
abstraction  hierarchy  and  decomposition,  these  basic  definitions  are  essential  to  their  uses  and 
applications.  Before  developing  any  further  modifications  or  applications  of  these  tools,  their 
fundamental  assumptions  and  theory  must  be  addressed  so  as  to  not  add  any  additions  to  a  house 
of  cards. 

The  first  definition  that  requires  analysis  is  that  of  the  nature  of  CWA,  which  is  advertised  as  a 
“formative”  design  approach  as  compared  to  descriptive  (designs  based  on  descriptive  models) 
or  normative  (designs  based  on  prescriptive  models).  Formative  models  are  defined  as  “A  model 
that  describes  requirements  that  must  be  satisfied  so  that  a  System  (sic)  could  behave  in  a  new, 
desired  way  (Vicente,  1999).”  This  definition  is  problematic  because  models  do  not  “describe” 
requirements.  Models  generalize  interrelations  from  observed  and/or  simulated  data,  ultimately 
to  predict  endogenous  variables  as  a  function  of  exogenous  variables.  While  there  are  many 
different  ways  to  model  (words,  mathematics,  diagrams,  etc.),  tractable  models  can  only 
represent  interrelations  of  a  small  set  of  variables,  and  thus  the  usefulness  of  a  model  is  typically 
inversely  proportional  to  the  number  of  variables  (Sheridan,  2005).  Since  models  are  general 
and  abstract  representations,  at  best  models  can  aid  an  engineer  in  developing  requirements. 
Models  do  not  map  directly  onto  the  development  of  requirements,  especially  detailed. 
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In  addition,  abstraction  decompositions  and  hierarchies  are  not  models.  These  are 
representations  of  system  elements  and  architectures,  but  they  fundamentally  lack  the  ability  to 
predict  one  or  more  exogenous  variables.  Jens  Rasmussen,  the  originator  of  these  tools, 
classifies  them  as  a  “framework  for  analysis  and  representation  aimed  at  eliminating  degrees  of 
freedom  in  the  set  of  behavior-shaping  constraints...  [which  allows]  converging  on  action 
alternatives  (Rasmussen,  Pejtersen,  &  Goodstein,  1994).”  He  further  refers  to  the  abstraction 
decomposition,  a  means-ends/part-whole  representation,  as  a  map  for  understanding  how,  what, 
and  why  a  system  is  used. 

The  use  of  the  abstraction  decomposition/hierarchy  to  represent  and  map  systems  has  been  used 
extensively  with  varying  degrees  of  success,  but  arguably  it  can  be  helpful  in  aiding  designers 
attempting  to  understand  a  complex  system.  However,  a  map  representation  is  not  the  same  as  a 
model,  and  both  engineers  and  psychologists  should  be  more  careful  in  applying  terminology 
that  is  not  appropriate.  Because  abstraction  decompositions  are  the  backbone  of  CWA,  it  is  not  a 
modeling  tool,  but  rather  is  a  domain  analysis  and  potential  system  architecture  mapping  tool 
which  will  be  discussed  below.  Again,  this  discussion  is  not  meant  to  trivialize  the  use  of  these 
tools  as  they  can  be  helpful  but  calling  them  models  is  simply  incorrect  and  misleading. 

Lastly,  a  discussion  on  the  term  “formative”  is  warranted.  CWA,  as  a  formative  approach  to 
design,  is  supposed  to  describe  requirements  that  MUST  be  met  so  that  a  system  can  behave  in 
some  predetermined,  more  effective  manner.  First,  it  is  not  at  all  clear  how  this  definition  is  so 
different  from  normative  since  it  is  prescriptive  as  well  (MUST).  In  addition,  this  definition 
further  assumes  that  CWA  analysts  know  what  the  “new,  desired  way”  is.  This  problem  is  not 
one  of  semantics,  especially  for  command  and  control  systems.  The  operation  of  a  nuclear 
power  plant  whose  goal  states  are  relatively  time-invariant  with  low  uncertainty  (e.g.,  make 
required  power  safely)  is  quite  different  from  that  of  a  command  and  control  network  in  which 
the  operations  are  not  only  highly  time-dependent,  but  also  subject  to  large  uncertainty, 
incomplete  knowledge  states,  and  changing  goals.  Any  analysis  approach  that  must  have  a 
defined  “new,  desired  way”  in  order  to  generate  requirements  cannot  be  effectively  applied  to 
command  and  control  systems. 

2.0  Requirements 

Any  single  analysis  approach  that  guarantees  comprehensive  requirements  is  dangerous. 
Requirements  generation  is  a  research  field  in  and  of  itself  (now  known  as  requirements 
engineering),  with  established  journals,  tools,  and  conferences.  It  is  not  a  simple  process  and 
cannot  be  adequately  addressed  either  by  a  single  tool  or  a  single  designer  or  group  of 
designers/engineers.  Standard  requirements  practice  contends  that  there  are  three  types  of 
system  requirements:  1)  functional  (what  the  equipment  must  do),  2)  nonfunctional  (performance 
measures),  and  3)  constraints  (the  system  limits.)  In  addition,  it  has  been  proposed  that  two 
categories  be  added,  that  of  human  performance  and  process  requirements  (Harrison  &  Forster, 
2003).  Other  human  factors  practitioners  have  developed  specific  methodologies  for 
requirements  generation  (Kirwan  &  Ainsworth,  1992;  Laughery,  1999;  Potter,  Elm,  Roth, 
Guallieri,  &  Easter,  2002;  Riley,  1992),  with  varying  degrees  of  success,  and  there  has  not  been  a 
generally  accepted  approach  for  cognitive  requirements  generation  within  the  larger  context  of 
systems  engineering. 
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It  has  been  asserted  that  CWA  can  generate  (or  describe)  human  systems  requirements,  however, 
this  is  a  point  of  debate.  For  example,  in  one  case  study,  use  of  the  CWA  provided  the  following 
functional  requirements  for  a  training  system  (Sanderson,  2003): 

•  Design  Objectives:  training  system  must  be  designed  to  satisfy  the  training  objectives  of 
the  work  domain 

•  Data  Collection:  training  system  must  be  capable  of  collecting  data  related  to  measures 
of  performance 

•  Scenario  Generation:  training  system  must  be  capable  of  generating  scenarios  for 
practicing  basic  training  functions 

•  Physical  Functionality:  training  systems  must  simulate  the  functionality  of  physical 
devices  and  significant  environmental  conditions 

•  Physical  Attributes:  training  system  must  recreate  functionally-relevant  properties  of 
physical  devices  and  significant  features  of  the  environment 

These  functional  requirements  as  generated  by  the  CWA  are  not  functional  requirements  as 
requirements  engineers  would  term  them  so  clearly  there  is  a  disconnect.  It  has  been  asserted 
that  CWA  tools  can  aid  in  the  generation  of  “information  requirements”  (Miller  &  Vicente, 

2001 ;  Potter,  Gualtieri,  &  Elm,  2002;  Vicente,  1999),  but  it  has  yet  to  be  established  how  and 
when  information  requirements  can  be  inserted  into  the  more  comprehensive  systems 
requirements  process.  Moreover,  it  has  been  shown  that  CWA  cannot  be  used  to  generate 
comprehensive  requirements  for  revolutionary  intentional  domain  systems13  (Cummings,  2003). 
In  addition,  as  will  be  discussed  in  a  subsequent  section,  it  is  not  clear  whether  or  not  CWA 
should  be  used  for  intentional  domains  that  are  time-dependent  with  high  degrees  of  uncertainty 
such  as  what  occurs  in  command  and  control  systems. 

3.0  CWA  and  Systems  Engineering 

Systems  engineering  is  not  a  mutually  exclusive  task  that  belongs  just  to  “systems  engineers.” 
The  systems  engineering  approach  recognizes  that  to  successfully  build  and  operationally  deploy 
a  system  (such  as  a  UAV,  a  ship,  or  even  a  command  and  control  network  ,  which  is  really  a 
system  of  systems)  a  principled  approach  must  be  taken  such  that  all  the  different  components  of 
the  system  are  seamlessly  integrated  in  final  design  stages  to  meet  customer  requirements.  The 
job  of  integrating  the  sub-systems  does  not  fall  just  to  systems  engineers  (who  are  often  system 
analysts)  and  program  managers,  but  also  to  any  engineer  of  any  background  who  will  integrate 
his/her  system  with  one  or  more  additional  systems. 


13  An  intentional  domain  is  one  in  which  human  intentions  constrain  the  systems  such  as  in  command  and  control 
systems,  versus  a  “causal  domain”  in  which  physical  laws  of  nature  constrain  the  system. 
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Figure  H-l:  The  Waterfall  Systems  Engineering  Approach 


By  the  very  nature  of  cognitive  engineering,  all  cognitive  engineers  should  be  “system 
engineers'”  in  that  the  human  component  is  always  integrated  with  multiple  layers  of  the  system. 
The  primary  purpose  of  a  cognitive  engineer  is  not  to  ensure  the  system  supports  the  human,  but 
that  the  human  is  effectively  integrated  into  the  system  such  that  overall  operational  success  can 
be  achieved. 

In  the  past,  military  systems  acquisition  typically  followed  a  waterfall  type  of  approach  as  seen 
in  Figure  H-l  (Blanchard  &  Fabrycky,  1998).  However,  recent  advances  in  systems  engineering 
approaches,  suggest  that  a  more  concurrent  approach  is  needed  both  for  a  leaner,  more  cost- 
effective  process  as  well  as  mitigation  of  risk.  This  approach  to  systems  engineering  is  known  as 


Figure  H-2:  The  Spiral  Systems  Engineering  Approach 
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the  Spiral  Model  (Figure  H-2,  (Boehm,  1988))  and  has  replaced  the  waterfall  model  in  most 
large  system  design  and  development  projects. 

While  CWA  takes  a  systems-theoretic  approach  in  potentially  determining  cognitive 
requirements  only  after  the  larger  system  is  mapped,  it  is  not  a  systems  engineering  approach, 
cognitive  or  otherwise.  Regardless  of  which  systems  engineering  model  is  used,  in  addition  to 
the  model  in  Figure  H-2  and  the  need  for  requirements  generation,  key  elements  of  systems 
engineering  include  concept  exploration,  demonstration  and  validation,  system  integration,  cost- 
benefit  analyses,  and  design  and  development  (Smootz,  2003).  CWA  does  not  address  any  of 
these  areas.  The  major  drawbacks  to  CWA  are  1)  no  definitive  link  to  design  (and  have  been 
routinely  criticized  for  such),  2)  a  lack  of  testing  and  verification  leverage  points,  and  other  than 
showing  vague  links  to  other  potential  supra  and  subsystems,  3)  no  information  is  given  towards 
effective  integration  strategies. 

In  terms  of  a  real  systems  engineering  model,  as  it  stands  now  with  its  five  phases,  CWA  is  at 
best  an  analysis  tool  for  human  systems  integration  information  requirements.  In  addition  to 
other  tools  such  as  legitimate  models,  computer  simulations,  and  more  traditional  cognitive  task 
analyses,  CWA  can  aid  in  identification  of  requirements  for  human-system  interaction.  This  is 
not  a  trivial  statement.  If  CWA  can  be  shown  to  reliably  and  clearly  delineate  those  information 
requirements  for  what,  how  much,  and  when  human  interaction  is  needed  in  a  system,  it  could 
revolutionize  the  human  systems  engineering  process. 

4.0  CWA  Limitations 

At  the  very  heart  of  the  CWA  methodology  is  the  use  of  the  abstraction-decomposition  matrix 
and  well  as  abstraction  hierarchies  (a  form  of  means-ends  analysis).  These  tools  in  theory 
represent  system  structure  at  different  levels  of  abstraction  so  that  a  system’s  functions  can  be 
decomposed  into  sub-systems.  This  function  decomposition  is  not  new  to  the  field  of  systems 
engineering  and  in  fact,  under  different  names,  has  been  in  use  much  longer  than  CWA  has  been 
in  existence.  The  following  quote  is  from  the  most  established  systems  engineering  text  in 
existence. 

“Functional  analysis  is  the  process  of  translating  system  requirements  into  detailed  design 
criteria,  along  with  the  identification  of  specific  resources  requirements  at  the  subsystem  level 
and  below.  One  starts  with  an  abstraction  of  the  needs  of  the  customer  and  works  down  to 
identify  the  requirements  for  hardware,  software,  people,  facilities,  data,  or  combinations  thereof 
(Blanchard  &  Fabrycky,  1998).” 

4. 1  Problems  with  Embedded  Control  Loops 

However,  where  CWA  differs  from  typical  system  engineering  functional  decompositions  is  the 
CWA  assertion  that  its  decompositions  represent  the  system  structure  independent  of  any  human, 
automation,  event,  or  task  goal.  A  significant  drawback  to  this  approach  is  that  embedded 
control  systems  cannot  be  represented  in  the  CWA  abstraction  decomposition  and  this  causes  the 
subsequent  representation  to  be  both  artificial  and  likely  incorrect.  While  current  criticism  of  the 
CWA  flaw  has  been  directed  towards  its  application  to  process  control  (Lind,  2003, 2004),  this  is 
a  criticism  that  is  even  more  applicable  in  the  command  and  control  domain.  There  are  countless 


embedded  control  loops  found  at  all  levels  of  C2  such  as  autopilot  in  planes,  GPS  navigation, 
electronic  intelligence,  radar-tracking  solutions  for  fire-support,  etc.  Military  platforms  and  C2 
networks  cannot  exist  without  embedded  control  loops  and  with  network-centric  warfare  on  the 
immediate  horizon,  the  presence  of  embedded  control  loops  will  only  increase. 

4.2  Adaptation  to  Revolutionary  Systems 

While  advertised  as  a  way  to  design  revolutionary  decision  support  systems,  CWA  can  only  be 
applied  to  existing  systems  (Cummings  &  Guerlain,  2003).  CWA  assumes  an  existing 
organizational  structure,  existing  infrastructure,  existing  users,  and  clearly  defined  boundaries. 
For  decision  support  systems  in  revolutionary  systems,  CWA  cannot  be  effectively  used  without 
other  cognitive  task  analysis  methodologies  such  as  cognitive  walkthroughs  and  simulations. 

4.3  Problems  with  Intentional  Domains 

CWA  has  been  shown  to  be  effective  for  analyzing  the  human  role  in  causal  systems  such  as 
process  control  (Vicente,  1999),  but  it’s  usefulness  in  intentional  domains  has  been  questioned 
(Cummings,  2003;  Wong,  Sallis,  &  O’Hare,  1998).  Especially  critical  to  command  and  control 
domains,  the  cause-and-effect  relationships  due  to  unanticipated  events  cannot  be  traced  via  the 
structural  invariants  provided  by  CWA.  In  addition,  as  mentioned  previously,  CWA  has  serious 
difficulty  in  addressing  the  concept  of  time  in  systems  operations.  In  the  first  phase  of 
abstraction  decomposition,  the  analyst  spends  a  great  deal  of  time  mapping  out  what,  how,  and 
why  relationships.  While  this  may  be  effective  for  causal  systems  whose  operations  do  not 
change  dramatically  over  time,  this  is  a  limitation  of  CWA  that  makes  its  application  to 
command  and  control  (intentional)  systems  extremely  limited,  as  is  the  inability  of  CWA  to 
incorporate  cause-and-effect  tracings  for  unanticipated  events  in  intentional  domains.  Any 
functional  or  design  requirement  that  comes  from  a  CWA  analysis  for  a  command  and  control 
system  should  be  suspect  since  there  is  no  principled  way  within  the  analysis  to  address  the 
critical  time  constraints. 

4. 4  Ill-defined  Phases  of  Analysis 

CWA  is  a  laborious  process  and  the  last  three  of  its  five  phases  are  very  vaguely  defined. 

Indeed,  many  researchers  and  analysts  will  only  complete  the  first  two  phases  and  call  their 
results  a  CWA.  Specifically  the  analysis  of  strategies,  social,  organizational,  and  cooperation 
analysis,  and  worker  competencies  phases  do  not  have  similar  principled  tools  that  are  used  in 
the  domain  and  control  task  analysis  phases.  Despite  numerous  tools  that  could  be  applied  to 
these  areas  (social  network  theory,  social  judgment  theory,  decision  trees,  macrocognition, 
human  performance  modeling,  etc.),  it  is  not  clear  how  and  if  these  phases  can  provide  any  real 
contribution  to  the  analysis  of  a  system  and  the  generation  of  any  meaningful  requirements  for 
decision  support. 

5.0  Structure  vs.  Process 

In  the  proposal  there  is  a  desire  to  separate  structure  and  process.  For  causal  systems  this  may  be 
possible  but  for  C2  systems  which  are  inherently  process  driven  this  may  not  be  possible.  In  the 
proposal,  structure  is  defined  as  the  instantiation  of  physical  resources  in  a  workspace.  In  C2  the 
overarching  purpose  of  the  system  is  to  move  the  structure  as  the  environment  dictates.  C2  is 
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fundamentally  a  process  and  this  may  be  yet  another  reason  why  CWA  is  not  applicable  to  C2. 
Even  Rasmussen  warns  that  functional  decomposition  according  to  structural  elements  cannot 
occur  because  of  human  adaptation  (Rasmussen  et  al.,  1994). 

Perhaps  the  distinction  should  be  made  between  structure  of  decision  support  interfaces  as 
opposed  to  physical  resources.  For  example,  how  an  interface  is  laid  out  can  be  structure  and  the 
processes  are  the  functionalities  supported  such  as  communications,  health  and  status 
monitoring,  introducing  new  targets  etc.  In  this  case  structure  should  be  guided  by  basic 
cognitive  and  usability  principles  in  conjunction  with  an  understanding  of  process  outcomes  and 
goals.  Unfortunately  at  this  low  level,  CWA  is  not  as  useful  as  cognitive  tasks  analysis  tools. 

6.0  Conclusions 

CWA  as  it  stands  now  seems  to  be  an  effective  tool  for  designers  who  need  to  develop  abstract 
representation  or  maps  of  a  complex  system.  One  potential  CWA  application  that  is  not 
addressed  explicitly  is  that  indeed  the  main  strength  of  CWA  may  be  to  help  designers  elicit 
knowledge  from  subject  matter  experts.  This  is  not  a  trivial  benefit.  It  is  often  very  difficult  for 
cognitive  engineers  to  grasp  all  the  nuances  of  human  interaction  in  complex  systems  so  if  the 
CWA  methodology  aids  the  practitioner  in  both  identifying  critical  relationships  and  mapping 
them  in  relation  to  other  systems,  then  this  is  a  valuable  tool.  However,  caution  must  be  taken  in 
that  such  linear  two-dimensional  representations  can  cause  researchers  to  miss  critical 
relationships  (a  hammer  looking  for  a  nail). 

While  CWA  has  merit  in  some  areas,  this  report  has  identified  four  major  problem  areas  in  the 
application  of  the  CWA  methodology  to  command  and  control  systems:  1)  embedded  system 
representation,  2)  application  to  intentional  domains,  3)  adaptation  to  revolutionary  systems,  and 
4)  ill-defined  phases  of  analysis.  In  addition,  the  disconnect  between  CWA  and  systems 
engineering  was  discussed  and  is  particularly  problematic  in  translating  any  information 
requirements  to  established  functional  and  design  requirements,  within  the  larger  systems 
engineering  process. 

The  following  recommendations  highlight  specific  areas  of  potential  exploration: 

•  Formalizing  a  human  systems  integration  requirements  generation  tool  that  could  include 
CWA  or  elements  thereof. 

•  Significantly  more  research  is  needed  in  better  defining  the  last  three  phases  of  CWA  and 
demonstrating  how  they  add  overall  value  to  the  requirements  process. 

•  Development  of  a  more  principled  application  of  CWA  as  a  knowledge  elicitation  and 
representation  tool. 

•  Formalize  a  methodology  in  which  CWA  can  be  shown  to  provide  consistent  information 
requirements  that  can  translate  to  display  design. 
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BACKGROUND 


Dr.  Peter  Kugler  developed  this  commentary  on  Dr.  Missy  Cummings’  critique  of  “Forms  of 
Representation  for  Cognitive  Demands  in  System  Acquisition”  (Appendix  H)  for  Dr.  Lintem 
under  a  subcontract  to  General  Dynamics.  Dr.  Kugler  is  a  private  consultant  with  extensive 
experience  in  the  foundations  of  cognitive  science,  computer  science,  and  artificial  intelligence. 
He  is  a  Visiting  Professor  at  the  University  of  Connecticut’s  Center  for  Ecological  Psychology. 
In  recent  years,  he  has  turned  his  attention  to  the  foundational  issues  of  cognitive  engineering  as 
it  has  become  evident  that  the  challenges  faced  by  cognitive  engineers  parallel  those  faced  by 
cognitive  scientists. 


DISCUSSION 

There  are  a  number  of  comments  that  I  want  to  make  on  a  number  of  different  levels. 

1.  First,  I  am  struck  by  the  difference  in  the  way  I  think  from  the  way  Missy  thinks.  For  me, 
first  and  foremost,  is  the  issue  of  “how  things  work”  (Box  1).  This  does  not  necessarily  require 
an  explicit  statement  about  how  things  work  but  rather  it  typically  appears  implicitly  in  the 
manner  of  posing  questions  and  guiding  discussions.  Posing  questions  can  be  as  important  (or 
more  important)  then  answering  questions.  I  do  not  see  any  new  question  being  posed  by  Missy. 
When  she  poses  a  question  it  is  about  the  appropriateness  of  Cognitive  Work  Analysis  (CWA) 
rather  than  about  ‘nature’  or  about  the  design  of  a  complex  ‘socio-technological  system.’ 


Box  1:  Questions  about  How  things  work 

Regardless  of  how  complex  a  system  is,  I  have  a  fundamental  belief  that  there  is  a  way  to 
inquire  and  talk  about  ‘how  this  system  works’  in  terms  of  fundamental  design  principles 
(versus  analysis  methodologies) — we  may  not  know  the  principles  but  our  goal  is  to  discover 
them  and  to  learn  how  to  use  them  in  the  service  of  better  and  safer  engineering  designs.  I 
want  to  know  how  the  system  is  constrained  and  what  classes  of  constraints  are  used.  I  want 
to  know  what  provides  an  informational  basis  for  system  control  and  how  this  information  is 
measured.  I  want  to  know  how  rigidly  designed  the  measurement  interface  is  and  if  there  are 
there  any  regions  in  the  control  system  requiring  an  ‘open  measurement  interface,’  e.g.  a 
human  observer,  for  ‘out-of -the-box’  solutions  to  potential  (or  unforeseen)  control  situations. 
I  also  want  to  know  how  the  human  controller  solves  the  control  problem  in  these  out-of-the- 
box  control  regions.  Jens  Rasmussen’s  CWA  methodology  was  directed  precisely  at  this  last 
question  but  also  touches  upon  many  of  the  earlier  questions  (Rasmussen  et  al.,  1994). 


2.  CWA  is  a  methodological  tool  that  can  help  organize  the  multitude  of  structural  and 
functional  descriptions  that  populate  complex  socio-technological  systems.  CWA  is  not  a 
‘model’  in  the  traditional  sense.  When  Kim  Vicente  (1999)  uses  the  word  ‘model’  in  the  context 
of  his  discussion  of  formative,  descriptive  and  normative  models,  the  context  makes  it  clear  that 
the  word  ‘model’  refers  to  a  more  general  class  of  models  than  the  limited  class  of  ‘computable 
models,’  including  those  referred  to  by  Sheridan  (2005)  that  involve  explicit  scaling  relations 
between  exogenous  and  endogenous  variables.  At  this  point  (of  reference)  in  Kim  V’s  book  the 
CWA  is  meant  to  provide  a  qualitative  mapping  (with  some  embedded  quantitative  relations 
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within  certain  cells)  that  is  similar  to  the  mapping  that  a  menu  provides  with  reference  to  food 
served  in  a  restaurant.  It  is  not  clear  if  Missy  wants  a  model  to  be  predictive  in  the  sense  that 
scaling  relations  are  both  computable  and  predictive — my  view  on  this  issue  is  that  most 
intentional  systems  are  non-computable  and  we  must  consider  the  criterion  of  conventional 
prediction  carefully  before  we  pose  it  as  a  requirement.  That  requirement  might  in  fact  eliminate 
“life  itself’  as  a  class  of  phenomenon  that  would  come  under  the  umbrella  of  the  criterion.  I 
think  Jens  [Rasmussen]  was  well  aware  of  this  possibility  and  that  is  why  he  did  not  require  this 
strong  form  of  prediction  as  a  requirement  of  his  methodology. 

3.  The  use  of  CWA  is  further  confused  when  Missy  suggests  that  it  is  inappropriate  for 
‘intentional  command  and  control  systems’  because  there  is  too  much  incomplete  knowledge, 
too  much  uncertainty,  and  too  many  changes  in  goal  states.  The  original  purpose  of  CWA  was  to 
help  to  understand  how  operators  control  a  system  when  they  depart  from  the  designed  control 
region,  i.e.  when  they  encounter  an  ‘out-of-the-box’  regime.  It  is  precisely  this  out-of-the-box 
region  that  has  the  same  ‘intentional  system  characteristics’  that  Missy  refers  to  with  her  term 
‘intentional  control  systems.’  I  agree  that  her  command-control  ‘dog  fighting’  between  pilots  is 

a  control  system  that  has  much  more  ‘openness’  in  terms  of  observables  and  ‘goal -purpose’  but 
both  of  these  solutions  involve  out-of-the-box  engineering  solutions.  Moreover,  these  types  of 
intentional  situations  lack  the  very  sense  of  prediction  that  Missy  critiques  with  reference  to 
Kim’s  use  of  ‘formative  model.’ 

4.  An  additional  recurrent  concern  of  Missy’s  is  how  CWA  relates  to  ‘Requirements 
Engineering,’  more  specifically  how  it  is  deficient  relative  to  the  standards  of ‘requirements 
engineering.’  She  presents  the  area  of  ‘requirements  engineering’  as  an  authoritative  information 
source,  when  in  fact  it  suffers  from  the  same  lack  of  clarity  that  all  discussions  do  when  the  topic 
has  to  do  with  the  topic  of  ‘functionality’  (or  ‘meaning’).  I  do  not  think  it  would  be  very 
difficult  for  me  to  talk  to  a  ‘requirements  engineer’  very  long  about  the  problem  of  how  to  define 
‘functionality’  or  ‘meaning’  before  they  would  agree  that  their  science  is  no  better  or  more 
precise  than  the  usage  being  advanced  in  CWA.  In  fact,  I  think  the  means-end  analysis  attempts 
to  address  this  issue  directly — I  applaud  CWA  analysts  for  this  direct  attempt.  But  this  is  an  area 
where  I  have  been  trying  to  drive  the  conceptual  wedge  concerning  the  problem  of  ‘context- 
sensitivity’  relative  to  functionality  or  meaning.  I  believe,  in  fact,  that  the  work  that  you  [Dr. 
Lintem]  and  I  have  been  doing  on  this  issue  far  surpasses  anything  I  have  seen  in  design 
engineering. 
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Background 


Dr.  Lintem  serves  Chief  Scientist  at  General  Dynamics  -  Advanced  Information  Engineering 
Services.  Dr.  Lintem  is  a  Subject  Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering. 
He  previously  served  as  Director  of  Human  Factors  for  the  Air  Operations  division  of  Australia’s 
Defense  Science  and  Technology  systems  research  where  he  managed  and  built  a  program  in 
Cognitive  Systems  Engineering.  Dr.  Lintem  reproduces  Dr.  Missy  Cummings’  critique  from 
Appendix  H  below,  interspersed  with  his  commentary  (underlined  italic  font)  at  relevant  points. 

Executive  Summary 

The  following  report  analyzes  Cognitive  Work  Analysis  (CWA),  with  special  emphasis  on 
command  and  control  decision  support  systems.  Specific  areas  of  investigation  include 
problematic  definitions  and  classifications  associated  with  CWA,  the  incompatibilities  of  CWA 
with  current  systems  engineering  practices  to  include  requirements  generations,  and  limitations 
of  CWA  to  include  1)  embedded  system  representation,  2)  application  to  intentional  domains, 

3)  adaptation  to  revolutionary  systems,  and  4)  ill-defined  phases  of  analysis.  A  section  is 
included  to  discuss  current  time  critical  targeting  issues  and  the  report  concludes  with  a  list  of 
recommendations  for  future  exploration. 

1.0  Definitions 

This  report  begins  with  a  discussion  of  definitions.  While  it  may  seem  a  trivial  and  semantic 
discussion  to  reexamine  definitions  surrounding  Cognitive  Work  Analysis  (CWA)  and  the 
abstraction  hierarchy  and  decomposition,  these  basic  definitions  are  essential  to  their  uses  and 
applications.  Before  developing  any  further  modifications  or  applications  of  these  tools,  their 
fundamental  assumptions  and  theory  must  be  addressed  so  as  to  not  add  any  additions  to  a  house 
of  cards. 

The  first  definition  that  requires  analysis  is  that  of  the  nature  of  CWA,  which  is  advertised  as  a 
“formative”  design  approach  as  compared  to  descriptive  (designs  based  on  descriptive  models) 
or  normative  (designs  based  on  prescriptive  models).  Formative  models  are  defined  as  “A  model 
that  describes  requirements  that  must  be  satisfied  so  that  a  System  (sic)  could  behave  in  a  new, 
desired  way  (Vicente,  1999).”  This  definition  is  problematic  because  models  do  not  “describe” 
requirements.  Models  generalize  interrelations  from  observed  and/or  simulated  data,  ultimately 
to  predict  endogenous  variables  as  a  function  of  exogenous  variables.  While  there  are  many 
different  ways  to  model  (words,  mathematics,  diagrams,  etc.),  tractable  models  can  only 
represent  interrelations  of  a  small  set  of  variables,  and  thus  the  usefulness  of  a  model  is  typically 
inversely  proportional  to  the  number  of  variables  (Sheridan,  2005).  Since  models  are  general 
and  abstract  representations,  at  best  models  can  aid  an  engineer  in  developing  requirements. 
Models  do  not  map  directly  onto  the  development  of  requirements,  especially  detailed. 

I  also  do  not  like  this  definition  of  formative.  I  do  not  even  think  it  captures  the  essence  of  Kim ’s 
discussion.  Mv  definition  of  Formative:  to  form  or  fashion  from  first  principles.  In  Cognitive 
Work  Analysis,  formative  implies  a  fashioning  on  the  basis  of  the  structure  of  the  work  system , 
such  as  the  functional  requirements  as  identified  through  the  analyses. 
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In  addition,  abstraction  decompositions  and  hierarchies  are  not  models.  These  are 
representations  of  system  elements  and  architectures,  but  they  fundamentally  lack  the  ability  to 
predict  one  or  more  exogenous  variables.  Jens  Rasmussen,  the  originator  of  these  tools, 
classifies  them  as  a  “framework  for  analysis  and  representation  aimed  at  eliminating  degrees  of 
freedom  in  the  set  of  behavior-shaping  constraints.. .[which  allows]  converging  on  action 
alternatives  (Rasmussen,  Pejtersen,  &  Goodstein,  1994).”  He  further  refers  to  the  abstraction 
decomposition,  a  means-ends/part-whole  representation,  as  a  map  for  understanding  how,  what, 
and  why  a  system  is  used. 

The  use  of  the  abstraction  decomposition/hierarchy  to  represent  and  map  systems  has  been  used 
extensively  with  varying  degrees  of  success,  but  arguably  it  can  be  helpful  in  aiding  designers 
attempting  to  understand  a  complex  system.  However,  a  map  representation  is  not  the  same  as  a 
model,  and  both  engineers  and  psychologists  should  be  more  careful  in  applying  terminology 
that  is  not  appropriate.  Because  abstraction  decompositions  are  the  backbone  of  CWA,  it  is  not  a 
modeling  tool,  but  rather  is  a  domain  analysis  and  potential  system  architecture  mapping  tool 
which  will  be  discussed  below.  Again,  this  discussion  is  not  meant  to  trivialize  the  use  of  these 
tools  as  they  can  be  helpful  but  calling  them  models  is  simply  incorrect  and  misleading. 

1  agree  that  the  use  of  model  in  cognitive  work  analysis  is  undisciplined — unfortunately,  this 
misuse  of  the  term  ‘model  ’  is  pervasive  in  behavioral  science. 

Lastly,  a  discussion  on  the  term  “formative”  is  warranted.  CWA,  as  a  formative  approach  to 
design,  is  supposed  to  describe  requirements  that  MUST  be  met  so  that  a  system  can  behave  in 
some  predetermined,  more  effective  manner.  First,  it  is  not  at  all  clear  how  this  definition  is  so 
different  from  normative  since  it  is  prescriptive  as  well  (MUST).  In  addition,  this  definition 
further  assumes  that  CWA  analysts  know  what  the  “new,  desired  way”  is.  This  problem  is  not 
one  of  semantics,  especially  for  command  and  control  systems.  The  operation  of  a  nuclear 
power  plant  whose  goal  states  are  relatively  time-invariant  with  low  uncertainty  (e.g.,  make 
required  power  safely)  is  quite  different  from  that  of  a  command  and  control  network  in  which 
the  operations  are  not  only  highly  time-dependent,  but  also  subject  to  large  uncertainty, 
incomplete  knowledge  states,  and  changing  goals.  Any  analysis  approach  that  must  have  a 
defined  “new,  desired  way”  in  order  to  generate  requirements  cannot  be  effectively  applied  to 
command  and  control  systems. 

/  cannot  agree  with  this — normative  and  formative  are  definitely  different  and  lead  to  different 
design  solutions.  Nevertheless,  this  disagreement  may  be  based  on  the  definition.  Possibly, 
upon  seeim  my  definition  she  may  agree  that  there  is  a  difference  between  formative  and 
normative. 


2.0  Requirements 

Any  single  analysis  approach  that  guarantees  comprehensive  requirements  is  dangerous.  Systems 
engineering  appears  to  make  this  claim  and  surely  this  is  essential  is  it  not?  Requirements 
generation  is  a  research  field  in  and  of  itself  (now  known  as  requirements  engineering),  with 
established  journals,  tools,  and  conferences.  It  is  not  a  simple  process  and  cannot  be  adequately 
addressed  either  by  a  single  tool  or  a  single  designer  or  group  of  designers/engineers.  Standard 
requirements  practice  contends  that  there  are  three  types  of  system  requirements:  1)  functional 
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(what  the  equipment  must  do),  2)  nonfunctional  (performance  measures),  and  3)  constraints  (the 
system  limits.)  I  actually  do  not  see  much  of  a  difference  between  Missy ’s  items  1)  and  3) — I 
think  of  functionality  and  constraint  as  a  duality.  In  addition,  it  has  been  proposed  that  two 
categories  be  added,  that  of  human  performance  and  process  requirements  (Harrison  &  Forster, 
2003).  Certainly  yes  to  process  requirements  but  human  performance  issues  should  also  be 
assessed  in  terms  of  functionality/constraints  and  process  requirements  and  also  performance 
measures.  In  my  view  there  is  a  troubling  attitude  in  the  engineering  and  design  community — 
there  is  a  tendency  to  believe  that  human  functionality  can  be  comfortably  replaced  with 
technological  functionality  coupled  with  the  belief  that  human  agents  should  be  assessed  in 
unique  ways.  1  believe  just  the  opposite ,  that  human  agents  should  be  assessed  on  the  same  types 
of  criteria  but  that  human  agents  have  the  capacity  for  functionality  that  cannot  he  instantiated 
by  technological  means — and  it  is  our  early  hard-core  engineers  and  designers  who  seem  to  fall 
into  this  trap  but  cognitive  engineers  often  seem  to  as  well.  Other  human  factors  practitioners 
have  developed  specific  methodologies  for  requirements  generation  (Kirwan  &  Ainsworth,  1 992; 
Laughery,  1999;  Potter,  Elm,  Roth,  Gualtieri,  &  Easter,  2002;  Riley,  1992),  with  varying  degrees 
of  success,  and  there  has  not  been  a  generally  accepted  approach  for  cognitive  requirements 
generation  within  the  larger  context  of  systems  engineering.  7  am  well  aware  of  the  work  by 
Laughery  and  by  Potter,  et  al.  and \  in  my  opinion,  their  statements  on  cognitive  requirements 
generation  are  entirely  unsatisfactory.  In  contrast.  I  do  believe  1  have  done  a  much  better  job  in 
the  work  on  Intelligence  Preparation  of  the  Battlesvace  (IPB)  that  I  finished  last  year  for  Dr. 

Bob  Eggleston  (AFRL/HECS).  In  addition ,  I  continue  to  work  on  that  issue  in  this  program  and 
in  our  internal  research  and  development  program. 

It  has  been  asserted  that  CWA  can  generate  (or  describe)  human  systems  requirements,  however, 
this  is  a  point  of  debate.  For  example,  in  one  case  study,  use  of  the  CWA  provided  the  following 
functional  requirements  for  a  training  system  (Sanderson,  2003): 

•  Design  Objectives:  training  system  must  be  designed  to  satisfy  the  training  objectives  of 
the  work  domain 

•  Data  Collection:  training  system  must  be  capable  of  collecting  data  related  to  measures 
of  performance 

•  Scenario  Generation:  training  system  must  be  capable  of  generating  scenarios  for 
practicing  basic  training  functions 

•  Physical  Functionality:  training  systems  must  simulate  the  functionality  of  physical 
devices  and  significant  environmental  conditions 

•  Physical  Attributes:  training  system  must  recreate  functionally-relevant  properties  of 
physical  devices  and  significant  features  of  the  environment 

These  functional  requirements  as  generated  by  the  CWA  are  not  functional  requirements  as 
requirements  engineers  would  term  them  so  clearly  there  is  a  disconnect.  At  first  glance,  this 
comment  is  something  of  a  puzzle  for  me.  What  would  constitute  functional  requirements  as 
requirements  engineers  would  term  them?  This  in  fact ,  is  one  of  the  central  issues  I  am  trying  to 
deal  with  in  this  project  and  in  our  I  RAD  flndeyendent  Research  And  Development 7  project.  In 
addition.  Vicente  fJ999.  page  115)  has  provided  a  table  that  lists  the  types  of  requirements  that 
can  be  derived  from  each  of  the  phases  of  Cognitive  Work  Analysis.  While  that  is  a  meager 
discussion ,  just  one  page  in  a  400-page  book,  it  offers  some  useful  ideas  we  can  build  on.  I  took 
this  comment  by  Missy  as  something  of  a  challenge  and  found  some  observations  towards  the 
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end  of  this  appendix  that  1  abstracted  and  expanded.  I  offer  those  ideas  separately  in 
Appendix  K.  It  has  been  asserted  that  CWA  tools  can  aid  in  the  generation  of  “information 
requirements”  (Miller  &  Vicente,  2001;  Potter,  Gualtieri,  &  Elm,  2002;  Vicente,  1999),  but  it  has 
yet  to  be  established  how  and  when  information  requirements  can  be  inserted  into  the  more 
comprehensive  systems  requirements  process.  Moreover,  it  has  been  shown  that  CWA  cannot  be 
used  to  generate  comprehensive  requirements  for  revolutionary  intentional  domain  systems16 
(Cummings,  2003).  In  addition,  as  will  be  discussed  in  a  subsequent  section,  it  is  not  clear 
whether  or  not  CWA  should  be  used  for  intentional  domains  that  are  time-dependent  with  high 
degrees  of  uncertainty  such  as  what  occurs  in  command  and  control  systems. 

The  real  value  of  CWA  is  with  intentional  domains,  but  1  am  not  sure  that  is  the  point  being  made 
here.  The  point  here  1  think  is  with  time  dependency — practically  everything  we  do  has  time 
dependency  characteristics.  I  and  others  have  explored  how  to  deal  with  that  in  the  Control 
Task  Analysis  phase  although  I  do  not  believe  this  issue  has  vet  been  resolved  effectively, 

3.0  CWA  and  Systems  Engineering 

Systems  engineering  is  not  a  mutually  exclusive  task  that  belongs  just  to  “systems  engineers.” 
The  systems  engineering  approach  recognizes  that  to  successfully  build  and  operationally  deploy 
a  system  (such  as  a  UAV,  a  ship,  or  even  a  command  and  control  network  ,  which  is  really  a 
system  of  systems)  a  principled  approach  must  be  taken  such  that  all  the  different  components  of 
the  system  are  seamlessly  integrated  in  final  design  stages  to  meet  customer  requirements.  The 
job  of  integrating  the  sub-systems  does  not  fall  just  to  systems  engineers  (who  are  often  system 
analysts)  and  program  managers,  but  also  to  any  engineer  of  any  background  who  will  integrate 
his/her  system  with  one  or  more  additional  systems. 
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Figure  J-l:  The  Waterfall  Systems  Engineering  Approach 


By  the  very  nature  of  cognitive  engineering,  all  cognitive  engineers  should  be  “system 
engineers”  in  that  the  human  component  is  always  integrated  with  multiple  layers  of  the  system. 
The  primary  purpose  of  a  cognitive  engineer  is  not  to  ensure  the  system  supports  the  human,  but 


16  An  intentional  domain  is  one  in  which  human  intentions  constrain  the  systems  such  as  in  command  and  control 
systems,  versus  a  “causal  domain”  in  which  physical  laws  of  nature  constrain  the  system. 


119 


that  the  human  is  effectively  integrated  into  the  system  such  that  overall  operational  success  can 
be  achieved. 

In  the  past,  military  systems  acquisition  typically  followed  a  waterfall  type  of  approach  as  seen 
in  Figure  J-l  (Blanchard  &  Fabrycky,  1998).  However,  recent  advances  in  systems  engineering 
approaches,  suggest  that  a  more  concurrent  approach  is  needed  both  for  a  leaner,  more  cost- 
effective  process  as  well  as  mitigation  of  risk.  This  approach  to  systems  engineering  is  known  as 
the  Spiral  Model  (Figure  J-2,  (Boehm,  1988))  and  has  replaced  the  waterfall  model  in  most  large 
system  design  and  development  projects. 


Figure  J-2:  The  Spiral  Systems  Engineering  Approach 


While  CWA  takes  a  systems-theoretic  approach  in  potentially  determining  cognitive 
requirements  only  after  the  larger  system  is  mapped,  (many  of  us  want  to  establish  cognitive 
requirements  before  the  larger  system  is  framed — we  want  to  influence  the  mapping  of  the  larger 
system)  it  is  not  a  systems  engineering  approach,  cognitive  or  otherwise  (so,  I  wonder  what 
constitutes  a  systems  engineering  approach — my  reading  of  systems  engineering  textbooks  does 
not  clarify  for  me  how  CWA  fails  in  this  resard ).  Regardless  of  which  systems  engineering 
model  is  used,  in  addition  to  the  model  in  Figure  J-2  and  the  need  for  requirements  generation, 
key  elements  of  systems  engineering  include  concept  exploration,  demonstration  and  validation, 
system  integration,  cost-benefit  analyses,  and  design  and  development  (Smootz,  2003).  CWA 
does  not  address  any  of  these  areas  (My  report  titled  “Contribution  of  Cognitive  Engineering  to 
Systems  Engineering"  (Appendix  K)  reflects  ideas  stimulated  by  Missy 's  comments.  However ,  / 
do  not  want  to  accept  the  position  that  a  cognitive  engineering  approach  needs  to  do  all  of  these: 
for  example,  1  believe  we  need  to  leave  cost-benefit  analysis  to  those  who  do  this  already).  The 
major  drawbacks  to  CWA  are  1)  no  definitive  link  to  design  (and  has  been  routinely  criticized 
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for  such)  (many  of  us  are  working  on  this  issue — and  on  2  below — 1  believe  1  have  made  some 
progress  on  the  link  to  design  in  my  IPB  work),  2)  a  lack  of  testing  and  verification  leverage 
points,  and  other  than  showing  vague  links  to  other  potential  supra  and  subsystems,  3)  no 
information  is  given  towards  effective  integration  strategies  ( whenever  I  hear  an  engineer  talk 
about  how  they  integrate  human  capabilities  with  technological  capabilities  I  am  struck  by  how 
superficial  and  unprincipled  their  approach  is — I  accept  that  this  is  something  we  need  to  work 
on  and  it  is  probably  something  that  only  a  cognitive  engineer  can  do). 

In  terms  of  a  real  systems  engineering  model,  as  it  stands  now  with  its  five  phases,  CWA  is  at 
best  an  analysis  tool  for  human  systems  integration  information  requirements.  Yes,  does  anyone 
imply  it  should  do  more?  In  addition  to  other  tools  such  as  legitimate  models,  computer 
simulations,  and  more  traditional  cognitive  task  analyses,  CWA  can  aid  in  identification  of 
requirements  for  human-system  interaction.  This  is  not  a  trivial  statement.  If  CWA  can  be 
shown  to  reliably  and  clearly  delineate  those  information  requirements  for  what,  how  much,  and 
when  human  interaction  is  needed  in  a  system,  it  could  revolutionize  the  human  systems 
engineering  process. 

4.0  CWA  Limitations 

At  the  very  heart  of  the  CWA  methodology  is  the  use  of  the  abstraction-decomposition  matrix 
and  well  as  abstraction  hierarchies  (a  form  of  means-ends  analysis).  These  tools  in  theory 
represent  system  structure  at  different  levels  of  abstraction  so  that  a  system’s  functions  can  be 
decomposed  into  sub-systems.  This  function  decomposition  is  not  new  to  the  field  of  systems 
engineering  and  in  fact,  under  different  names,  has  been  in  use  much  longer  than  CWA  has  been 
in  existence.  The  following  quote  is  from  the  most  established  systems  engineering  text  in 
existence. 

“Functional  analysis  is  the  process  of  translating  system  requirements  into  detailed  design 
criteria,  along  with  the  identification  of  specific  resources  requirements  at  the  subsystem  level 
and  below.  One  starts  with  an  abstraction  of  the  needs  of  the  customer  and  works  down  to 
identify  the  requirements  for  hardware,  software,  people,  facilities,  data,  or  combinations  thereof 
(Blanchard  &  Fabrycky,  1998).” 

There  are  two  issues  here — one  regarding  the  relationshiv  of  cosnitive  engineering  to  systems 
engineering  and  the  other  regarding  the  nature  of  functional  analysis.  I  address  the  first  issue 
and  offer  a  few  comments  on  the  second  in  a  separate  appendix  titled  “ The  Relationship  of 
Cognitive  Engineering  to  Systems  Engineering”  ( Appendix  D). 

The  CWA  approach  to  functional  analysis  might  be  clarified  by  consideration  of  the  meaning  of 
the  terms  function  and  process.  These  are  troubling  terms  in  engineering  and  science  because 
their  range  of  usage  is  broad  and  they  have  overlapping  meanings.  Within  Cognitive  Work 
Analysis,  Vicente  (1999)  has  given  them  constrained  meanings  that  map  onto  the  needs  of  this 
analytic  framework.  A  function  is  a  structural  property  of  the  work  domain.  A  process  is  a 
mechanism  by  which  the  behavior  of  the  system  is  produced.  This  distinction  is  unusual  and  no 
other  strategy  of  cognitive  analysis  makes  it  explicit.  An  underlying  assumption  of  Cognitive 
Work  Analysis  is  that  the  separation  of  structure  from  activity  helps  bring  an  important  source  of 
order  to  the  analysis  of  complex,  socio-technical  systems.  Even  within  Systems  Engineering, 
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where  this  sort  of  distinction  would  seem  to  offer  an  advantage.  Functional  Analysis  as  discussed 
in  many  texts  and  reports  (e.s.,  as  in  Blanchard  and  Fabrycky,  1990)  is  a  functional  flow 
analysis — essentially  a  process  analysis. 

4. 1  Problems  with  Embedded  Control  Loops 

However,  where  CWA  differs  from  typical  system  engineering  functional  decompositions  is  the 
CWA  assertion  that  its  decompositions  represent  the  system  structure  independent  of  any  human, 
automation,  event,  or  task  goal.  A  significant  drawback  to  this  approach  is  that  embedded 
control  systems  cannot  be  represented  in  the  CWA  abstraction  decomposition  and  this  causes  the 
subsequent  representation  to  be  both  artificial  and  likely  incorrect.  While  current  criticism  of  the 
CWA  flaw  has  been  directed  towards  its  application  to  process  control  (Lind,  2003,  2004),  this  is 
a  criticism  that  is  even  more  applicable  in  the  command  and  control  domain.  There  are  countless 
embedded  control  loops  found  at  all  levels  of  C2  such  as  autopilot  in  planes,  GPS  navigation, 
electronic  intelligence,  radar-tracking  solutions  for  fire-support,  etc.  Military  platforms  and  C2 
networks  cannot  exist  without  embedded  control  loops  and  with  network-centric  warfare  on  the 
immediate  horizon,  the  presence  of  embedded  control  loops  will  only  increase. 

1  am  not  sure  where  the  notion  that  we  should  embed  control  loops  in  the  abstraction- 
decomposition  may  came  from — I  never  do  it  and  1  am  never  tempted  to  do  it.  For  me  at  least, 
this  is  definitely  the  wrong  place  to  be  thinking  about  control  loops. 

4. 2  Adaptation  to  Revolutionary  Systems 

While  advertised  as  a  way  to  design  revolutionary  decision  support  systems,  CWA  can  only  be 
applied  to  existing  systems  (Cummings  &  Guerlain,  2003).  CWA  assumes  an  existing 
organizational  structure,  existing  infrastructure,  existing  users,  and  clearly  defined  boundaries. 

For  decision  support  systems  in  revolutionary  systems,  CWA  cannot  be  effectively  used  without 
other  cognitive  task  analysis  methodologies  such  as  cognitive  walkthroughs  and  simulations. 

We  definitely  want  to  assist  with  design  of  future  systems ,  but  this  talk  of  revolutionary  desisn 
has  been  undisciplined — any  desisn  has  to  come  from  a  hybrid  revolutionary-evolutionary 
approach.  Neelam  ’s  work  on  team  desisn  is  an  excellent  example  of  how  to  prosress  through  a 
revolutionary-evolutionary  cycle. 

4.3  Problems  with  Intentional  Domains 

CWA  has  been  shown  to  be  effective  for  analyzing  the  human  role  in  causal  systems  such  as 
process  control  (Vicente,  1999),  but  it’s  usefulness  in  intentional  domains  has  been  questioned 
(Cummings,  2003;  Wong,  Sallis,  &  O'Hare,  1998).  Especially  critical  to  command  and  control 
domains,  the  cause-and-effect  relationships  due  to  unanticipated  events  cannot  be  traced  via  the 
structural  invariants  provided  by  CWA.  In  addition,  as  mentioned  previously,  CWA  has  serious 
difficulty  in  addressing  the  concept  of  time  in  systems  operations.  In  the  first  phase  of 
abstraction  decomposition,  the  analyst  spends  a  great  deal  of  time  mapping  out  what,  how,  and 
why  relationships.  While  this  may  be  effective  for  causal  systems  whose  operations  do  not 
change  dramatically  over  time,  this  is  a  limitation  of  CWA  that  makes  its  application  to 
command  and  control  (intentional)  systems  extremely  limited,  as  is  the  inability  of  CWA  to 
incorporate  cause-and-effect  tracings  for  unanticipated  events  in  intentional  domains.  Any 
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functional  or  design  requirement  that  comes  from  a  CWA  analysis  for  a  command  and  control 
system  should  be  suspect  since  there  is  no  principled  way  within  the  analysis  to  address  the 
critical  time  constraints. 

Cause-effect  and  time  criticality  are  two  different  issues  and  they  are  mixed  up  here — 1  have 
addressed  time  constraints  above.  Regarding  cause-effect ,  it  is  not  entirely  clear  to  me  that  we 
should  be  mapping  out  cause-effect  in  cognitive  engineering — although  1  may  be  unclear  about 
what  is  meant  here  by  that.  Peter  Kugler  offered  some  thoughts  on  prediction  for  intentional 
systems  in  his  commentary  in  Appendix  I— those  comments  are  relevant  to  this  issue  of  cause- 
effect. 

4. 4  Ill-defined  Phases  of  A  nalysis 

CWA  is  a  laborious  process  and  the  last  three  of  its  five  phases  are  very  vaguely  defined. 

Indeed,  many  researchers  and  analysts  will  only  complete  the  first  two  phases  and  call  their 
results  a  CWA.  Specifically  the  analysis  of  strategies,  social,  organizational,  and  cooperation 
analysis,  and  worker  competencies  phases  do  not  have  similar  principled  tools  that  are  used  in 
the  domain  and  control  task  analysis  phases.  Despite  numerous  tools  that  could  be  applied  to 
these  areas  (social  network  theory,  social  judgment  theory,  decision  trees,  macrocognition, 
human  performance  modeling,  etc.),  it  is  not  clear  how  and  if  these  phases  can  provide  any  real 
contribution  to  the  analysis  of  a  system  and  the  generation  of  any  meaningful  requirements  for 
decision  support. 

These  phases  in  CWA  are  essential  but  I  agree  we  have  not  vet  done  a  good  job  on  them.  1 
suspect  that  the  fact  that  we  have  not  done  a  good  job  on  the  later  phases  leads  others  to  express 
concern  about  including  properties  in  the  work  domain  analysis  that  should  be  dealt  with  in  the 
later  phases. 

5.0  Structure  vs.  Process 

In  the  proposal  there  is  a  desire  to  separate  structure  and  process.  For  causal  systems  this  may 
be  possible  but  for  C2  systems  which  are  inherently  process  driven  this  may  not  be  possible.  In 
the  proposal,  structure  is  defined  as  the  instantiation  of  physical  resources  in  a  workspace.  In  C2 
the  overarching  purpose  of  the  system  is  to  move  the  structure  as  the  environment  dictates.  C2  is 
fundamentally  a  process  and  this  may  be  yet  another  reason  why  CWA  is  not  applicable  to  C2. 
Even  Rasmussen  warns  that  functional  decomposition  according  to  structural  elements  cannot 
occur  because  of  human  adaptation  (Rasmussen  et  al.,  1994). 

No.  structure  is  not  just  physical  resources — it  also  involves  functions  and  constraints — nothing 
we  do  is  fundamentally  process — there  is  always  some  form  of  supporting  structure  and  the 
claim  in  the  proposal  is  that  it  is  useful  to  map  that  out  as  distinct  from  process.  I  further  want 
to  arsue  that  we  need  to  focus  on  designing  structure.  Many  people  want  to  design  process,  and 
while  I  do  think  we  can  have  some  influence  there,  the  design  of  process  by  those  who  are  not 


17  The  document  that  Dr.  Cummings  critiqued,  “Forms  of  Representation  for  Cognitive  Demands  in  System 
Acquisition”  was  actually  the  proposal  that  Dr.  Lintem  submitted  to  AFRL/HECS  for  this  effort.  It  is  sometimes 
referred  to  as  the  “proposal”  in  this  appendix. 
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subject  matter  experts  carries  enormous  risk.  One  of  the  major  claims  out  of  cognitive  work 
analysis  is  that  we  need  to  have  operators  "complete  the  design.  ’’  This  means ,  we  build  the 
structure  and  they  decide  how  to  use  it  to  accomplish  the  purposes  for  which  the  system  was 
built.  We  can  help  and  suide  the  design  of  process  but  we  need  to  be  very  careful  that  we  do  not 
impose  clumsy  and  brittle  processes  on  those  who  must  use  the  system.  1  have  arsued  elsewhere 
that  this  imposition  of  clumsy  and  brittle  processes  on  workers  is  the  dominant  cause  of 
accidents  in  high-risk  systems. 

Perhaps  the  distinction  should  be  made  between  structure  of  decision  support  interfaces  as 
opposed  to  physical  resources.  For  example,  how  an  interface  is  laid  out  can  be  structure  and  the 
processes  are  the  functionalities  supported  such  as  communications,  health  and  status 
monitoring,  introducing  new  targets  etc.  In  this  case  structure  should  be  guided  by  basic 
cognitive  and  usability  principles  in  conjunction  with  an  understanding  of  process  outcomes  and 
goals.  Unfortunately  at  this  low  level,  CWA  is  not  as  useful  as  cognitive  tasks  analysis  tools. 

6.0  Conclusions 

CWA  as  it  stands  now  seems  to  be  an  effective  tool  for  designers  who  need  to  develop  abstract 
representation  or  maps  of  a  complex  system.  One  potential  CWA  application  that  is  not 
addressed  explicitly  is  that  indeed  the  main  strength  of  CWA  may  be  to  help  designers  elicit 
knowledge  from  subject  matter  experts.  This  is  not  a  trivial  benefit.  It  is  often  very  difficult  for 
cognitive  engineers  to  grasp  all  the  nuances  of  human  interaction  in  complex  systems  so  if  the 
CWA  methodology  aids  the  practitioner  in  both  identifying  critical  relationships  and  mapping 
them  in  relation  to  other  systems,  then  this  is  a  valuable  tool.  However,  caution  must  be  taken  in 
that  such  linear  two-dimensional  representations  can  cause  researchers  to  miss  critical 
relationships  (a  hammer  looking  for  a  nail). 

While  CWA  has  merit  in  some  areas,  this  report  has  identified  four  major  problem  areas  in  the 
application  of  the  CWA  methodology  to  command  and  control  systems:  1)  embedded  system 
representation,  2)  application  to  intentional  domains,  3)  adaptation  to  revolutionary  systems,  and 
4)  ill-defined  phases  of  analysis.  In  addition,  the  disconnect  between  CWA  and  systems 
engineering  was  discussed  and  is  particularly  problematic  in  translating  any  information 
requirements  to  established  functional  and  design  requirements,  within  the  larger  systems 
engineering  process. 

The  following  recommendations  highlight  specific  areas  of  potential  exploration: 

•  Formalizing  a  human  systems  integration  requirements  generation  tool  that  could  include 
CWA  or  elements  thereof. 

•  Significantly  more  research  is  needed  in  better  defining  the  last  three  phases  of  CWA  and 
demonstrating  how  they  add  overall  value  to  the  requirements  process. 

•  Development  of  a  more  principled  application  of  CWA  as  a  knowledge  elicitation  and 
representation  tool. 

•  Formalize  a  methodology  in  which  CWA  can  be  shown  to  provide  consistent  information 
requirements  that  can  translate  to  display  design. 
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APPENDIX  K.  CONTRIBUTION  OF  COGNITIVE  ENGINEERING  TO  SYSTEMS 

ENGINEERING 
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BACKGROUND 


In  this  appendix,  Dr.  Lintem  discusses  ideas  stimulated  by  his  reading  of  the  report  from  Dr. 
Missy  Cummings  (see  Appendix  J).  Dr.  Cummings’  ideas  are  restated  in  italics  in  the  discussion 
below. 

Dr.  Lintem  serves  Chief  Scientist  at  General  Dynamics  -  Advanced  Information  Engineering 
Services.  Dr.  Lintem  is  a  Subject  Matter  Expert  in  the  field  of  Cognitive  Systems  Engineering. 
He  previously  served  as  Director  of  Human  Factors  for  the  Air  Operations  division  of  Australia’s 
Defense  Science  and  Technology  systems  research  where  he  managed  and  built  a  program  in 
Cognitive  Systems  Engineering. 


DISCUSSION 

CWA  is  a  systems-theoretic  approach,  not  a  systems  engineering  approach  (a  point  John  Flach 
keeps  repeating,  we  possibly  have  not  been  sufficiently  explicit  about  this  in  general). 

Systems  Engineering  focuses  on  requirements  generation.  We  need  to  support  that  focus  but  we 
do  not  currently  do  a  good  job  of  requirements  generation.  We  need  to  work  on  that. 

Important  Challenges,  in  addition  to  requirements  generation,  how  can  Cognitive  Engineering 
help  with: 

•  Concept  Exploration  (primarily  the  Abstraction-Decomposition  (A-D)  matrix) 

•  Demonstration  (probably  not) 

•  Validation  (validation  of  what?) 

•  System  Integration  (yes,  with  Human-System  Integration) 

•  Cost-Benefit  Analyses  (marginal  contribution) 

•  Design  and  Development  (yes,  with  the  Human-System  Integration  elements  of  Design 
and  Development) 

Need  to  work  on: 

•  Definitive  link  to  design  (we  are  working  this  issue  and  have  made  progress) 

•  Testing  and  verification  leverage  points  (we  can  contribute  to  this) 

•  Effective  integration  strategies  (we  can  develop  these) 

Other  tools  such  as  models,  computer  simulations,  and  more  traditional  cognitive  task  analyses 
can  aid  in  identification  of  requirements  for  human-system  integration,  (yes,  no  doubt  about 
this — and  in  testing  design  options) 

If  CWA  can  be  shown  to  reliably  and  clearly  delineate  those  information  requirements  for  what, 
how  much,  and  when  human  interaction  is  needed  in  a  system,  it  could  revolutionize  the  human 
systems  engineering  process ,  e.g., 

•  By  laying  out  functional  requirements  in  the  Abstraction-Decomposition  matrix  and  then 
identifying  what  needs  to  be  accomplished  within  the  system,  Cognitive  Work  Analysis 
takes  the  first  step  towards  identifying  staffing  requirements,  i.e.,  one  or  more  agents 
need  to  be  allocated  to  each  module  of  work — although,  at  this  stage,  there  is  no 
resolution  of  whether  they  should  be  machine  agents  or  human  agents. 
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•  Control  Task  Analysis  and  Strategies  Analysis  will  identify  the  ways  in  which  things  can 
be  accomplished — however,  to  fully  resolve  the  machine-agent  versus  human-agent 
allocations,  it  is  necessary  to  integrate  the  results  of  these  analyses  with  insights  from 
studies  of  interactions  between  humans  and  automation. 

The  primary  purpose  of  a  cognitive  engineer  is  not  to  ensure  the  system  supports  the  human,  but 
that  the  human  is  effectively  integrated  into  the  system  such  that  overall  operational  success  can 
be  achieved. 

I  have  a  mixed  response  to  this  comment — I  agree  that  the  system  is  not  designed  to  support  the 
human  participants  and  so  in  that  sense  it  is  misleading  to  say  that  our  goal  is  to  ensure  the 
system  supports  the  human,  but  my  specific  problem  with  the  second  part  of  this  statement — 
ensure  that  the  human  is  effectively  integrated  into  the  system — is  that  it  accords  priority  to  the 
system  so  that  humans  might  be  thought  of  as  supporting  the  system.  The  priority  is  to  satisfy 
the  (human-related)  purposes  for  which  the  system  is  designed;  the  purposes  that  constitute 
operational  success. 

This  is  similar  to  the  problem  I  have  with  Morten  Lind  (Appendix  G)  when  he  argues  that  human 
operators  need  to  understand  the  system.  While  I  agree  with  this  claim,  the  statement  on  its  own 
ignores  the  prior  requirement  to  design  a  system  so  that  it  is  compatible  with  the  robust  cognitive 
style  of  experienced  operators,  thus  enabling  the  understanding  of  the  system  by  the  operators. 

I  typically  refer  to  requirements  for  Human-System  Integration,  which  seems  to  carry  the  right 
message.  The  focus  of  cognitive  engineering  is  restricted  to  the  relationship  between  human 
participants  and  the  system  technology,  and  we  need  to  ensure  that  the  technology  is  compatible 
at  that  seam  with  a  robust  human  cognitive  style. 

As  cognitive  engineers,  we  need  to  confine  our  requirements  specifications  to  human-system 
integration  issues,  but  as  shown  by  the  table  below  from  Vicente  (1999)  there  is  at  least  some 
thinking  in  this  area.  I  accept  that  it  is  a  neglected  area  but  it  is  one  in  which  at  least  some  of  us 
are  making  progress. 

RELATIONSHIPS  BETWEEN  THE  FIVE  PHASES  OF  CWA  AND  VARIOUS  CLASSES 
OF  SYSTEMS  DESIGN  INTERVENTIONS.  (TABLE  5.1  from  page  115,  Vicente  1999) 

1.  Work  Domain 

What  information  should  be  measured?  (sensors) 

What  information  should  be  derived?  (models) 

How  should  information  be  organized?  (database) 


18  Vicente,  K.  (1999).  Cognitive  Work  Analysis.  Toward  safe,  productive,  and  healthy  computer-based  work. 
Mahwah,  NJ:  Erlbaum. 
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2.  Control  Tasks 

What  goals  must  be  pursued  and  what  are  the  constraints  on  those  goals?  (procedures  or 
automation) 

What  information  and  relations  are  relevant  for  particular  classes  of  situations?  (context  sensitive 
interface) 

3.  Strategies 

What  frames  of  reference  are  useful?  (dialog  modes) 

What  control  mechanisms  are  useful?  (process  flow) 

4.  Social-Organizational 

What  are  the  responsibilities  of  all  of  the  actors?  (role  allocation) 

How  should  actors  communicate  with  each  other?  (organizational  structure) 

5.  Worker  Competencies 

What  knowledge,  rules,  and  skills  do  workers  need  to  have?  (selection,  training,  and  interface 
form) 


SUMMARY 

For  me,  the  most  valuable  part  of  Missy's  report  was  the  commentary  on  the  nature  of  systems 
engineering.  The  primary  concern  with  requirements  generation,  and  then  subsequently  with 
Concept  Exploration,  Demonstration,  Validation,  System  Integration  and  Cost-Benefit  Analyses 
forced  me  to  ponder  first,  whether  we  are  doing  a  good  job  on  any  of  these  dimensions  and 
second,  how  we  might  develop  our  capability  on  the  dimensions  to  which  we  should  be  able  to 
contribute.  I  take  this  as  a  reasonable  statement  of  an  agenda  for  this  project  and  for  any  effort  in 
which  we  are  trying  to  integrate  cognitive  engineering  with  systems  engineering. 
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